-
Notifications
You must be signed in to change notification settings - Fork 27
Avoid deadlocks after fork #303
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,157 @@ | ||
| /******************************************************************************** | ||
| * Copyright (c) 2026 Contributors to the Eclipse Foundation | ||
| * | ||
| * See the NOTICE file(s) distributed with this work for additional | ||
| * information regarding copyright ownership. | ||
| * | ||
| * This program and the accompanying materials are made available under the | ||
| * terms of the Apache License Version 2.0 which is available at | ||
| * https://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| ********************************************************************************/ | ||
|
|
||
| #include <limits.h> | ||
|
|
||
| #include <libgen.h> | ||
| #include <sys/resource.h> | ||
| #include <cerrno> | ||
| #include <cstring> | ||
| #include <string> | ||
| #include <string_view> | ||
|
|
||
| #include "score/launch_manager/src/daemon/src/osal/return_types.hpp" | ||
| #include "score/mw/launch_manager/osal/security_policy.hpp" | ||
| #include "score/mw/launch_manager/osal/set_affinity.hpp" | ||
| #include "score/mw/launch_manager/osal/set_groups.hpp" | ||
| #include "score/mw/launch_manager/process_group_manager/iprocess.hpp" | ||
|
|
||
| /* | ||
| * In this file only async signal safe functions can be used, as these functions | ||
| * are used between `fork` and `execve`. | ||
| * | ||
| * This is necessary because `fork` only copies the current thread, so any locks | ||
| * which were held at that time will never be released. See `man 2 fork`. | ||
| * | ||
| * A big implication is that we must use assertions rather than log messages, | ||
| * and the assertion handler itself must follow these rules. | ||
| */ | ||
|
|
||
| namespace score | ||
| { | ||
|
|
||
| namespace lcm | ||
| { | ||
|
|
||
| namespace internal | ||
| { | ||
|
|
||
| void setLimit(const int resource, const std::size_t amount, const std::string_view rlimit_name) | ||
| { | ||
| if (amount == 0) | ||
| return; | ||
|
|
||
| const struct rlimit limit{ | ||
| .rlim_cur = amount, | ||
| .rlim_max = amount, | ||
| }; | ||
|
|
||
| const auto result = ::setrlimit(resource, &limit); | ||
| SCORE_LANGUAGE_FUTURECPP_ASSERT_PRD_MESSAGE( | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. bazel assort macros terminate the process using std::abort(). Is this safe to terminate the process this way btw. fork() and execve()?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think here it is not so much about the signal safety, but other considerations what is being cleaned up: https://www.unixguide.net/unix/programming/1.1.3.shtml Not sure though exactly what is the difference btw. _exit and std::abort, but If I remember correctly using _exit() is the recommended way to terminate a process btw. fork and execve in case of error |
||
| result != -1, ("Failed to set rlimit " + std::string(rlimit_name) + ": " + std::strerror(errno)).c_str()); | ||
| } | ||
|
|
||
| void setSchedulingAndSecurity(const osal::OsalConfig& config) | ||
| { | ||
| int result; // Used for return value from system calls | ||
|
|
||
| // Set process group id to be equal to the pid | ||
| // setpgid will fail if called by a session leader (which LCMd is), so skip | ||
| if (config.comms_type_ != osal::CommsType::kLaunchManager) | ||
| { | ||
| result = setpgid(0, getpid()); | ||
| SCORE_LANGUAGE_FUTURECPP_ASSERT_PRD_MESSAGE(result == 0, std::strerror(errno)); | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think the strategy to use asserts here is now conflicting with this #299 Do you have any suggestion how to resolve this?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We could allocate a fixed buffer for the assertion handler and call
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It sounds like a good solution to me. What do you think @paulquiring ?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We need something like this as assertion handler. void my_signal_handler(int sig) {
char buffer[128];
// Format your string (snprintf is generally safe in practice though technically not guaranteed)
int len = snprintf(buffer, sizeof(buffer), "Caught signal %d\n", sig);
// write to stdout (file descriptor 1) is async-signal-safe
write(STDOUT_FILENO, buffer, len);
}
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. snprintf is also not async signal safe 😅
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @NicolasFussberger and I discussed this a bit more see #303 (comment) |
||
| } | ||
|
|
||
| // Set scheduling policy with sched_setscheduler | ||
| sched_param sch_param{}; | ||
|
|
||
| sch_param.sched_priority = config.scheduling_priority_; | ||
|
|
||
| // TODO: Clamping should be done when the config is loaded | ||
|
danth marked this conversation as resolved.
|
||
| // https://github.com/eclipse-score/lifecycle/issues/304 | ||
| if (sch_param.sched_priority < sched_get_priority_min(config.scheduling_policy_)) | ||
| { | ||
| sch_param.sched_priority = sched_get_priority_min(config.scheduling_policy_); | ||
| } | ||
| else if (sch_param.sched_priority > sched_get_priority_max(config.scheduling_policy_)) | ||
| { | ||
| sch_param.sched_priority = sched_get_priority_max(config.scheduling_policy_); | ||
| } | ||
|
|
||
| result = sched_setscheduler(0, config.scheduling_policy_, &sch_param); | ||
| SCORE_LANGUAGE_FUTURECPP_ASSERT_PRD_MESSAGE(result != -1, std::strerror(errno)); | ||
|
|
||
| result = osal::setaffinity(config.cpu_mask_); | ||
| SCORE_LANGUAGE_FUTURECPP_ASSERT_PRD_MESSAGE(result != -1, std::strerror(errno)); | ||
|
|
||
| result = setgid(config.gid_); | ||
| SCORE_LANGUAGE_FUTURECPP_ASSERT_PRD_MESSAGE(result != -1, std::strerror(errno)); | ||
|
|
||
| // Note: the type of the first parameter of setgroups() differs in Linux and QNX, so we use osal | ||
| const auto gids_size = config.supplementary_gids_.size(); | ||
| if (gids_size > 0) | ||
| { | ||
| const auto gids_data = config.supplementary_gids_.data(); | ||
| result = osal::setgroups(gids_size, gids_data); | ||
| SCORE_LANGUAGE_FUTURECPP_ASSERT_PRD_MESSAGE(result != -1, std::strerror(errno)); | ||
| } | ||
|
|
||
| result = setuid(config.uid_); | ||
| SCORE_LANGUAGE_FUTURECPP_ASSERT_PRD_MESSAGE(result != -1, std::strerror(errno)); | ||
| } | ||
|
|
||
| void changeCurrentWorkingDirectory(const osal::OsalConfig& config) | ||
| { | ||
| // Notice that this next static variable is duplicated by the fork() and so does not need | ||
| // any protection by a mutex although at first sight you may think it could need one. | ||
| static char path_copy[PATH_MAX + 1U] = {0}; | ||
|
|
||
| // TODO: This should be validated when the config is loaded | ||
|
NicolasFussberger marked this conversation as resolved.
|
||
| // https://github.com/eclipse-score/lifecycle/issues/304 | ||
| SCORE_LANGUAGE_FUTURECPP_ASSERT_PRD_MESSAGE(config.executable_path_.size() < PATH_MAX, | ||
| "Executable path is too long"); | ||
|
|
||
| const auto result = chdir(dirname(strncpy(path_copy, config.executable_path_.c_str(), PATH_MAX))); | ||
| SCORE_LANGUAGE_FUTURECPP_ASSERT_PRD_MESSAGE(result != -1, std::strerror(errno)); | ||
| } | ||
|
|
||
| void implementMemoryResourceLimits(const osal::OsalConfig& config) | ||
| { | ||
| setLimit(RLIMIT_DATA, config.resource_limits_.data_, "RLIMIT_DATA"); | ||
| setLimit(RLIMIT_AS, config.resource_limits_.as_, "RLIMIT_AS"); | ||
| setLimit(RLIMIT_STACK, config.resource_limits_.stack_, "RLIMIT_STACK"); | ||
|
|
||
| // Note about cpu limit: | ||
| // Using setrlimit, this imposes a maximum time that a process will run for, which might not be | ||
| // what you intend? Probably you'll want a maximum time in a time-slice, but you don't get that | ||
| // with limits set by setrlimit... | ||
| setLimit(RLIMIT_CPU, config.resource_limits_.cpu_, "RLIMIT_CPU"); | ||
| } | ||
|
|
||
| void changeSecurityPolicy(const osal::OsalConfig& config) | ||
| { | ||
| if (config.security_policy_ == "") | ||
| return; | ||
|
|
||
| const auto result = osal::setSecurityPolicy(config.security_policy_.c_str()); | ||
| SCORE_LANGUAGE_FUTURECPP_ASSERT_PRD_MESSAGE( | ||
| result == 0, | ||
| ("Failed to set security policy " + config.security_policy_ + ": " + std::strerror(errno)).c_str()); | ||
| } | ||
|
|
||
| } // namespace internal | ||
|
|
||
| } // namespace lcm | ||
|
|
||
| } // namespace score | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| /******************************************************************************** | ||
| * Copyright (c) 2026 Contributors to the Eclipse Foundation | ||
| * | ||
| * See the NOTICE file(s) distributed with this work for additional | ||
| * information regarding copyright ownership. | ||
| * | ||
| * This program and the accompanying materials are made available under the | ||
| * terms of the Apache License Version 2.0 which is available at | ||
| * https://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| ********************************************************************************/ | ||
|
|
||
| #include <string_view> | ||
|
|
||
| #include "score/mw/launch_manager/osal/security_policy.hpp" | ||
| #include "score/mw/launch_manager/process_group_manager/iprocess.hpp" | ||
|
|
||
| namespace score | ||
| { | ||
|
|
||
| namespace lcm | ||
| { | ||
|
|
||
| namespace internal | ||
| { | ||
|
|
||
|
danth marked this conversation as resolved.
|
||
| /// @brief Sets the limit if given a non-zero value, otherwise skips. | ||
| /// @warning This will abort if not successful. | ||
| void setLimit(const int resource, const std::size_t amount, const std::string_view rlimit_name); | ||
|
|
||
| /// @warning This will abort if not successful. | ||
| void setSchedulingAndSecurity(const osal::OsalConfig& config); | ||
|
|
||
| /// @warning This will abort if not successful. | ||
| void changeCurrentWorkingDirectory(const osal::OsalConfig& config); | ||
|
|
||
| /// @warning This will abort if not successful. | ||
| void implementMemoryResourceLimits(const osal::OsalConfig& config); | ||
|
|
||
| /// @warning This will abort if not successful. | ||
| void changeSecurityPolicy(const osal::OsalConfig& config); | ||
|
|
||
| } // namespace internal | ||
|
|
||
| } // namespace lcm | ||
|
|
||
| } // namespace score | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we have the same async signal safe issue with the assert message, as the string concatenation may cause heap allocation which is not async signal safe
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could remove the extra info and just use
strerroras the message