Summarized using AI

Bringing Linux pidfd to Ruby

Maciej Mensfeld • April 16, 2025 • Matsuyama, Ehime, Japan • Talk

In the presentation titled 'Bringing Linux pidfd to Ruby' at RubyKaigi 2025, Maciej Mensfeld discusses the challenges of process and fork management in Ruby, which traditionally relies on Process IDs (PIDs). He notes that this can lead to race conditions and security vulnerabilities, particularly in high-throughput systems. The talk highlights the introduction of pidfd in Linux 5.3, which provides a more reliable method for managing processes than conventional PIDs, but notes the absence of native Ruby support for pidfd.

Key Points Discussed:
- Introduction to the Problem:
- Maciej emphasizes the difficulties that arise in Ruby when using PIDs for process management, such as PID reuse leading to potential errors and race conditions when managing child processes.
- He mentions specific challenges like signal race conditions and difficulties with tracking process trees, common issues when processes continue to fork and manage workload in multi-threaded environments.

  • Pidfd as a Solution:

    • Introduced in Linux 5.3, pidfd utilizes file descriptors for referencing processes, reducing issues such as PID reuse and allowing for more stable signal handling. It allows reliable and non-blocking monitoring of process states.
    • The speaker elaborates on how pidfd's design eliminates many of the race conditions typically seen with PID management.
  • Implementation in Ruby with FFI:

    • Lacking native support, Mensfeld demonstrates how he used Foreign Function Interface (FFI) in Ruby to construct an abstraction over the pidfd APIs, thus incorporating these features into Ruby applications like Karafka and Shyuken.
    • He explains the challenges and successes in adapting these features, including how to handle system call mappings and manage child processes effectively.
  • Lessons Learned:

    • Mensfeld emphasizes the complexity of robust process management and acknowledges the importance of utilizing the right tools and APIs to avoid security pitfalls in Ruby applications.
    • He expresses his intent to extract his findings into a standalone Ruby gem to help others implement these practices in their work.

Conclusion:
Mensfeld wraps up by reflecting on the need for Ruby to evolve and better support modern Linux features. He encourages the Ruby community to embrace these tools for improved reliability and security in process management. The presentation showcases both the technical depth required to manage processes efficiently in Ruby and the innovative approaches developers can take to overcome existing limitations.

Overall, attendees gained insights into advanced process management strategies using modern Linux features, enhancing the stability and security of Ruby applications managing concurrent processes.

Bringing Linux pidfd to Ruby
Maciej Mensfeld • Matsuyama, Ehime, Japan • Talk

Date: April 16, 2025
Published: May 27, 2025
Announced: unknown

Processes and fork management in Ruby have traditionally relied on PIDs, leading to race conditions and potential security issues in high-throughput systems. While Linux 5.3 introduced pidfd as a solution, Ruby lacks native support for this feature. I'll demonstrate how I bridged this gap using FFI, exploring syscall mappings, zombie process prevention, and practical process management patterns. Through live demonstrations, you'll learn how Linux pidfd APIs can help you eliminate race conditions and make your Ruby applications more reliable in modern environments.

https://rubykaigi.org/2025/presentations/maciejmensfeld.html

RubyKaigi 2025

00:00:01.520 hello
00:00:03.919 good morning all of the jetlacked people
00:00:06.919 uh nice seeing you all of you
00:00:10.599 uh okay let's get started uh welcome to
00:00:13.599 my bringing Linux PD to Ruby talk
00:00:18.920 uh before we start I wanted to say that
00:00:21.600 I'm really happy to see so many people
00:00:23.840 interested in this and I know you're not
00:00:26.560 totally not here because it's the only
00:00:28.160 English track at the
00:00:30.599 moment my name is Mati Mensfeld i come
00:00:33.440 from Poland i'm a software architect and
00:00:37.120 I've been doing Ruby for half of my life
00:00:39.920 pretty much
00:00:41.600 uh I work at a security company where I
00:00:44.320 deal with open source supply chain and
00:00:47.280 uh LLM security i'm the creator of this
00:00:51.680 funny framework called Karafka for
00:00:53.760 working with Apache Kafka and I I don't
00:00:55.600 know if there's anyone using it but I
00:00:57.280 hope there is here um I'm the newly
00:01:01.760 appointed maintainer of Shyuken which is
00:01:04.680 a SQS equivalent of Karafka if you could
00:01:08.159 say i adopted it from from Pablo a few
00:01:11.439 weeks
00:01:12.200 ago it doesn't have a
00:01:15.000 logo and I'm a Ruby gem security team
00:01:19.000 member although we're not going to talk
00:01:21.200 about security much today uh we're going
00:01:23.439 to talk about uh challenges in process
00:01:27.360 management in Ruby i'm going to talk
00:01:30.080 about some weird solutions to some of
00:01:32.479 those problems i'm going to show you
00:01:34.479 some implementation details of how I
00:01:36.960 tried to solve them and I'm going to
00:01:39.920 talk about what I would like to achieve
00:01:42.960 this year in regards to this topic
00:01:47.079 uh I'm here because I love Ruby to be
00:01:51.640 honest it's not the best language ever
00:01:54.320 because I don't think there's one that
00:01:55.920 is the best but I think that this one is
00:01:57.680 pretty dope uh I also love Ruby Kai it's
00:02:02.560 my third time speaking here and each
00:02:05.040 time I come uh to Rubik Kai I'm like
00:02:07.280 super excited and super happy mostly
00:02:09.920 about the
00:02:11.560 afterparties
00:02:13.160 uh so if you see me at the after parties
00:02:16.879 just make sure to come and talk with me
00:02:18.720 i will have some sweets from Poland so
00:02:20.879 if you want them well you don't have an
00:02:23.840 option not
00:02:25.000 to but I'm also not
00:02:27.959 delusional and I'm well aware that uh in
00:02:32.480 our work we need to use the right tools
00:02:34.879 uh to do the job and the thing is Ruby
00:02:36.959 is not always the right tool to do
00:02:39.519 certain things but at the same time I
00:02:42.640 really believe that we can make we we
00:02:46.319 can make Ruby the right tool to do way
00:02:49.599 more than we do now and I don't mean
00:02:51.519 that by like by trying to force Ruby to
00:02:55.120 you do things it shouldn't be doing but
00:02:58.400 rather expanding the space of
00:03:01.080 capabilities uh
00:03:04.120 hence what I want to talk about today uh
00:03:07.040 the challenge that I had with both like
00:03:09.519 Karaf Kashuk and few other things I'm
00:03:11.519 working on uh the things that I build
00:03:14.720 are usually multi-threaded heavily
00:03:16.560 multi-threaded uh 10 20 30 threads uh
00:03:19.920 doing weird stuff and trying to
00:03:21.360 coordinate their own
00:03:23.400 reality and uh sometimes I stumble on
00:03:27.120 the
00:03:28.200 JVL uh probably we all
00:03:31.239 do and uh usually it isn't a problem for
00:03:34.879 for people like me building background
00:03:37.200 processing engines uh you could say I
00:03:40.319 could just ignore it right because uh a
00:03:43.519 lot of the work that is happening is
00:03:45.360 happening uh with a lot of IO
00:03:50.120 so most of the people here will know
00:03:52.560 that okay if it's IO bound it's not a
00:03:54.720 problem to run with threads but there's
00:03:57.000 always this space of CPU intense tasks
00:04:00.640 and
00:04:02.840 uh what is interesting is that users of
00:04:06.959 once in a while they uh misunderstand
00:04:10.640 what is IO bound and CPUbound and by
00:04:13.599 root that I don't know if byoot is here
00:04:15.599 but uh hi man he wrote a great article
00:04:20.400 about it and he this is a really good
00:04:24.479 quote about that that
00:04:27.160 uh
00:04:28.840 if
00:04:30.520 the majority of stuff we're doing would
00:04:33.280 actually be heavily IO bound wouldn't be
00:04:35.919 as good as it is with what it's doing
00:04:38.880 and the same applies to what I do what I
00:04:41.919 try to achieve uh and even in in
00:04:45.120 background processing and stuff like
00:04:47.040 that there's
00:04:48.880 IO is only a fraction of of things that
00:04:52.240 Ruby is doing with a lot of things being
00:04:56.320 done on the CPU side uh so what I wanted
00:05:00.240 to do is I wanted to be bring a CPU
00:05:02.880 friendly parallelization to Karafa and
00:05:05.440 show Yukand and part of it is management
00:05:07.520 of processes
00:05:11.320 uh I wanted to do it with RTORS
00:05:13.759 initially right the almost new shiny
00:05:17.199 tool that we have uh they actually fit
00:05:20.080 really well with my reality with what I
00:05:22.960 do uh much better than
00:05:26.520 uh
00:05:28.120 forks and I tried to do it uh I built
00:05:32.160 some PC's I did some benchmarking and so
00:05:35.039 on uh but
00:05:37.160 then they're experimental that's one of
00:05:40.240 the things the second thing is uh we
00:05:42.320 already have something for utilizing
00:05:45.199 multiple cores which is processes and
00:05:47.759 they're production
00:05:49.320 proven uh but nonetheless I still wanted
00:05:53.520 to uh test RAS and it's not a RTOR talk
00:05:59.000 uh and I but I did right if and if they
00:06:02.560 would work I wouldn't be here uh or
00:06:05.039 maybe I would be talking about their
00:06:07.280 usage but uh as of now for the stuff
00:06:10.479 that I am doing they're just slowing
00:06:14.000 things down I have really good use cases
00:06:16.639 for them but unfortunately the way they
00:06:18.639 are uh they're not helping much
00:06:21.560 so I had to resolve the
00:06:25.000 forks which is like quite okay you I
00:06:28.000 came up with this rather boring and
00:06:30.000 typical design for having a supervision
00:06:34.080 process from which I could fork out my
00:06:37.440 uh worker notes for Karafka and
00:06:41.000 Shyuken workers would do their work
00:06:43.919 pretty simple and the supervisor would
00:06:46.560 like check if everything is fine and
00:06:48.560 would occasionally kill processes or uh
00:06:52.639 fork more processes and so on nothing
00:06:55.759 nothing fancy no and
00:06:59.560 uh it it's kind of funny because I build
00:07:02.560 gems but at the same time I don't want
00:07:04.400 to use gems of other people because I
00:07:06.639 see them as a liability to my users uh
00:07:10.639 maybe that's because I deal with supply
00:07:12.319 chain issues in Ruby gems
00:07:16.319 uh so I wanted to to keep keep it uh
00:07:18.880 simple
00:07:20.039 stupid and obviously I did right the
00:07:23.280 easiest implementation you can have uh
00:07:26.520 for management of processes is to just
00:07:29.280 fork things collect the pits and when
00:07:32.639 you have them you can do stuff with them
00:07:35.319 uh really simple stuff actually because
00:07:38.319 there's like not a lot of things
00:07:40.319 happening uh that's what I
00:07:43.400 thought you need to stop them once in a
00:07:45.840 file so if you decide to stop them you
00:07:47.919 can just send them a
00:07:49.479 signal and uh that's pretty much that
00:07:53.440 you can wait for them and there's a bit
00:07:57.120 of supervision that is irrelevant to
00:07:59.840 this talk and it worked and it was like
00:08:03.919 a decent implementation for a PC uh so I
00:08:07.919 created a a pull request to
00:08:10.280 Karafka and I sent it to KJ from uh Ruby
00:08:14.000 core team unfortunately he is not here
00:08:16.960 at Ruby Kai this year but yeah if if if
00:08:20.960 you guys know him and I know there's
00:08:22.879 some people here that know him he has
00:08:25.080 this amazing talent of making you refing
00:08:29.919 your career in it because he can have 20
00:08:33.680 comments to a single line of code so
00:08:37.680 obviously what I did I showed it to him
00:08:39.839 because you know I like to make things
00:08:42.159 hard for myself
00:08:45.080 uh and alongside many other things that
00:08:49.440 he pointed out uh that made made me
00:08:52.480 really angry at myself
00:08:55.880 he made this uh comment that PFDs are
00:09:00.560 the absolutely are absolutely the right
00:09:02.959 professional API and I am a professional
00:09:05.120 programmer right so I need to use the
00:09:07.440 professional
00:09:08.440 APIs uh gosh
00:09:12.920 yeah uh but I had no idea what the hell
00:09:16.000 is that
00:09:17.720 i grabbed the Ruby dogs for PDS nothing
00:09:22.160 was there right uh I' I've started
00:09:24.880 reading about what it is what problems
00:09:27.839 uh are we trying to solve with them and
00:09:31.200 uh yeah there are quite few problems
00:09:34.880 uh that I was not aware maybe you guys
00:09:37.760 are but I I wasn't aware of them uh so
00:09:41.600 the first problem that I've encountered
00:09:43.360 is a pit reuse uh pits are just numbers
00:09:46.720 they're nothing special they're just
00:09:49.519 numbers they're limited on every single
00:09:52.560 machine you have certain pool of them
00:09:55.279 they can run out and then when they run
00:09:58.000 out they're recycled meaning a new
00:10:00.399 process can get the same pit that uh
00:10:03.839 your process that died
00:10:06.519 had and this can create really funny
00:10:10.040 cases when you're building supervisors
00:10:13.200 or monitoring tools
00:10:16.120 uh because one process may die and a
00:10:20.079 different absolutely not related process
00:10:22.320 may get the same bit and if you're not
00:10:25.600 cautious
00:10:27.480 uh your software may not even notice
00:10:33.240 that and
00:10:35.160 uh it got me thinking that it it
00:10:39.000 actually I did the same uh and I think
00:10:42.240 each of us killed a process once in a
00:10:44.320 while kills a process once in a while
00:10:46.079 right uh we have a misbehaving puma or
00:10:49.200 whatever running with eating up 100% of
00:10:52.160 CPU not uh reacting to command C we
00:10:56.320 identify it uh we find its speed we send
00:10:59.760 a signal and there's no process at all
00:11:02.000 because it died uh and the same race
00:11:06.240 condition could happen to a Ruby code
00:11:09.959 right it may try to send signals to a
00:11:13.279 process that just died and well what
00:11:16.079 what what
00:11:17.880 what about it uh well it's just sending
00:11:22.160 signals to a wrong process right but the
00:11:24.320 thing is it's not only sending signals
00:11:26.600 uh it may cause a lot of problems and
00:11:30.480 some of them are related to security
00:11:33.600 so there were vulnerabilities discovered
00:11:37.440 that were related to exactly this uh
00:11:41.279 that would allow some types of software
00:11:43.200 to
00:11:44.440 uh escape out of sandboxes or have
00:11:48.720 escalated
00:11:50.200 privileges uh and well by accident you
00:11:53.839 could all also kill minus 9 a new
00:11:56.480 process that was critical uh and you
00:11:58.959 probably wouldn't even notice that
00:12:03.120 and
00:12:03.959 uh I have some code and I have some
00:12:06.399 examples i'm going to put them on GitHub
00:12:07.920 after this talk and uh you can run it
00:12:11.040 yourself just run it in a virtual
00:12:12.959 machine because it's a bit aggressive
00:12:14.880 and I killed my machine a few times uh
00:12:18.160 but you you can run code that will
00:12:21.360 demonstrate
00:12:23.480 uh how you can hijack the the the pit
00:12:29.320 uh and it actually runs really fast like
00:12:32.639 I run a virtual machine i have some Ruby
00:12:34.959 code that tries that spuns a lot of
00:12:37.440 processes that kill kill kills one with
00:12:40.480 a pit 424 and then it attempts to catch
00:12:44.079 this pit again by aggressively forking
00:12:46.959 new processes and it in this example it
00:12:50.320 took me like 348 attempts before I got
00:12:53.360 the same pit ID right obviously the
00:12:55.920 bigger the pit spaces the less frequent
00:12:58.320 it's going to happen uh but you know we
00:13:00.639 run a lot of software all the time so it
00:13:03.120 will happen once in a
00:13:08.279 while but that's only one of the
00:13:10.480 problems the second problem is uh
00:13:13.040 signals race conditions
00:13:15.519 uh when you you're you you're building
00:13:18.160 especially gems not end user
00:13:20.760 software you can either build the
00:13:23.680 execution environment which is the space
00:13:26.800 where other people's code runs right you
00:13:29.920 do some stuff you fetch data and then
00:13:31.600 you give it to the end user you yield it
00:13:34.000 and then once the end user is done with
00:13:36.160 their work you do something else and
00:13:38.880 then the cycle repeats this is pretty
00:13:40.800 much my uh life of my open
00:13:43.880 sources Or you can build libraries right
00:13:46.720 they don't yel they provide some of the
00:13:48.880 capabilities and those realms sometimes
00:13:51.600 may collide with each other uh because
00:13:54.240 Ruby signal handlers are global per
00:13:57.160 process so what that means
00:14:00.279 is you may end up with a race conditions
00:14:03.839 related to
00:14:05.480 forking uh because when you create a
00:14:09.839 child process and it dies it stops but
00:14:13.279 dice is a bad word it finishes its work
00:14:16.839 uh
00:14:19.560 gladly Linux will Unix will send a sick
00:14:22.560 child back to the parent process and
00:14:28.519 uh this means that when it sends it uh
00:14:32.639 whoever calls a wait bit to get to
00:14:36.160 collect the exit code will win and if
00:14:39.680 you have
00:14:41.320 few places in the code that are trying
00:14:44.079 to do the same because one is from the
00:14:47.199 execution environment and the second is
00:14:49.040 from the library uh only one can win
00:14:52.000 right so if let's say this is me and
00:14:55.920 Karafka I set up a signal because I'm
00:14:57.839 interested in in in it i wait for any
00:15:01.720 process child process to die so I can
00:15:05.040 collect and log status and take some
00:15:06.880 actions right uh but at the same time
00:15:09.360 there's someone building a library that
00:15:11.440 is forking as
00:15:13.240 well and trying to collect statuses
00:15:16.240 because this library is also interested
00:15:19.120 in utilizing forks they will
00:15:21.680 fundamentally collide with each other
00:15:24.079 what that means is only one of them can
00:15:26.560 collect the uh exit code and it's either
00:15:30.320 going to be the signal handler or it's
00:15:32.160 going to be the library right that's
00:15:35.959 uh that but that's a problem that bit
00:15:39.199 fds are not going to solve but I'm
00:15:42.000 pointing that out to you guys just so
00:15:45.120 you're aware of it
00:15:48.680 uh but there's one more issue there
00:15:52.160 actually two one is like in general
00:15:54.800 signal handing and the third problem is
00:15:57.759 uh being limit limitations of sik child
00:16:01.839 and limitations of
00:16:04.680 uh the pit approach so you basically
00:16:08.759 cannot track processes trees if your
00:16:12.160 it's like a a bit like a dysfunctional
00:16:14.240 family because if your child has a child
00:16:18.079 uh you as a grandparent will not know
00:16:20.000 about it and if this child dies you will
00:16:22.160 not know about it as
00:16:23.880 well and uh well
00:16:26.759 if the parent dies the grandchild is
00:16:30.880 being orphaned but then it's being
00:16:32.959 adopted by in it not by uh the
00:16:35.600 grandparent
00:16:37.279 and also there's no way I mean it's it's
00:16:40.320 tricky but there's not a nice way maybe
00:16:43.440 no way is a bad word there's no nice way
00:16:45.279 to monitor parents from children's
00:16:48.160 perspective right there are no automatic
00:16:50.480 signals nothing like that and uh yeah
00:16:54.399 that's pretty much this it's uh
00:16:56.720 grandparent dies only uh parent receives
00:17:00.399 a signal
00:17:05.679 Which means that if you decide to build
00:17:07.839 supervisor software in Ruby or anything
00:17:10.799 of that type
00:17:13.319 your first idea is probably is going to
00:17:17.199 be to send uh zero signals to processes
00:17:22.160 and as long as you have permissions to
00:17:23.919 send signals uh it's not going to crash
00:17:27.799 and your um you're going to know that a
00:17:31.360 process is running you will not know if
00:17:33.360 it's the same process uh that you're
00:17:37.120 going to check in a second because of
00:17:38.720 the p we use but at least you're going
00:17:40.400 to know that something is alive right uh
00:17:43.280 so I I briefly checked who's using bits
00:17:46.320 in Ruby and everyone is using them uh
00:17:49.360 because it's a nice API right except
00:17:53.520 Samuel Williams I don't know if Samuel
00:17:55.760 is here uh but IO events has hooks for
00:18:00.080 uh PV
00:18:02.520 Uh but there's this beautiful API in
00:18:06.320 Linux called process file
00:18:08.840 descriptor that was added to Linux uh 6
00:18:12.160 years
00:18:13.080 ago uh and it provides a stable
00:18:15.760 reference to a process meaning it's not
00:18:19.080 a semi- random number that may or may
00:18:22.880 not point you to a certain process is a
00:18:25.520 file descriptor that is that will always
00:18:29.200 point you to the process uh to the same
00:18:32.240 process even if it dies uh and it solves
00:18:35.520 two of those three problems that I
00:18:37.520 talked
00:18:38.360 about uh for for people that don't know
00:18:41.440 file descriptor is just a way of uh it's
00:18:44.640 a Unix abstraction for resources
00:18:47.760 management if you want any type of
00:18:49.360 resources files sockets pipes or
00:18:51.440 anything like that uh and you ask the
00:18:53.760 operating system it's going to create uh
00:18:56.720 an abstraction it's going to assign a
00:18:58.480 number to this resource and it's going
00:18:59.919 to give you this number so we can use it
00:19:04.200 um as I said the primary benefit is that
00:19:07.440 it provides a stable reference to a
00:19:09.440 process it's immune to speed reuse it's
00:19:12.919 pullable uh meaning you can create
00:19:15.919 complex non-blocking uh eventbased EO
00:19:20.080 based uh software that can monitor any
00:19:23.679 type of processes not only processes
00:19:26.240 within your uh family tree and it
00:19:30.080 provides race free signal deliveries
00:19:33.760 meaning if you dispatch a signal to a
00:19:35.440 process it's going to go to this
00:19:37.760 particular process and only to this
00:19:39.919 process
00:19:42.200 simple simple as that uh it has only few
00:19:46.880 a uh core APIs from Linux perspective uh
00:19:51.039 you can request a descriptor for a given
00:19:54.919 process you can send a signal using a
00:19:58.880 file
00:20:00.039 descriptor uh and you can clean up and
00:20:03.679 collect the terminated processes aside
00:20:06.799 from that you can use typical IO uh IO
00:20:11.200 based APIs because it's a file
00:20:13.280 descriptor right the fact that it is
00:20:15.120 also a process file descriptor
00:20:17.919 uh has nothing to do with it uh but Ruby
00:20:21.039 has no PD support it has only pit
00:20:24.679 support uh so what can we do about
00:20:27.480 it there was a feature proposal by KJ
00:20:30.640 where he mentioned P FD as part of the
00:20:33.360 feature proposal uh it didn't went
00:20:35.919 through and to be honest I don't know
00:20:37.600 why uh I think it was a really weird
00:20:40.559 feature proposal and PFD was just there
00:20:43.679 by maybe by accident but you can easily
00:20:45.919 use an FFI abstraction layer with uh and
00:20:49.280 this is what I used for Karafka
00:20:52.559 uh because PDFs are absolutely the right
00:20:56.720 professional
00:20:59.640 API using it is easy you take FFI you
00:21:03.120 import lip See uh and you attach the
00:21:06.480 functions that I mentioned right uh it's
00:21:09.520 basically just sys
00:21:11.000 call after sys call and weight ID and uh
00:21:15.120 the only thing you you really need to
00:21:16.799 figure out at this stage is what are the
00:21:18.880 sysol numbers because all of the sys
00:21:21.039 call uh operations are identified by
00:21:24.200 numbers and there is a beautiful website
00:21:26.960 to do that that gives you all of it for
00:21:30.159 all of the platforms luckily for us uh
00:21:33.520 the sys calls that we are we are we we
00:21:35.679 were interested I was interested in uh
00:21:38.320 have the same numbers across all of
00:21:40.640 almost all of the uh
00:21:43.000 architectures um but if you if you
00:21:46.320 decide to do stuff with this table just
00:21:49.360 make sure that you provide users with
00:21:51.200 ability to change it if they run their
00:21:54.159 software on really weird platforms
00:21:57.880 uh because it's Linux only I had to do
00:22:00.640 some detection and it turns out that BSD
00:22:03.440 actually has some of those APIs
00:22:06.520 uh but it doesn't have the PD support so
00:22:09.919 I use a really funny trick i create I
00:22:13.440 try to create a
00:22:14.679 PDF during
00:22:17.080 the for myself and if it fails I just
00:22:19.919 assume okay this platform doesn't
00:22:21.360 support it
00:22:23.400 um and once you once you have those APIs
00:22:26.159 you can easily request a pet
00:22:28.840 FD as long as doesn't fail it doesn't
00:22:31.760 fail you have it and then you can wrap
00:22:34.240 it with an IO object right so we can do
00:22:36.480 all of this regular nice IO stuff like
00:22:39.600 uh you can check if it's readable if the
00:22:43.039 file descriptor is readable
00:22:46.799 uh it means your process is dead if it's
00:22:49.919 not readable it means it's
00:22:52.720 uh if it's readable there's nothing to
00:22:55.200 read but this is the way of of checking
00:22:58.320 uh whether the process is alive or not
00:23:02.240 you can send signals the same way almost
00:23:05.360 the same way as you would send normal
00:23:06.880 signals but in a consistent and reliable
00:23:09.960 way you provide the sys call
00:23:12.919 you provide the signal and the file
00:23:16.880 descriptor right and uh and you can
00:23:20.080 collect zombie processes the processes
00:23:22.080 that finished but weren't yet
00:23:24.440 collected and uh yeah with only those
00:23:30.720 uh Linux APIs you can build a really
00:23:33.679 nice I think you can build a really nice
00:23:35.840 layer uh for processes management
00:23:38.799 because you can check that the process
00:23:40.159 is alive you can signal it and you can
00:23:42.000 collect uh the process or do the cleanup
00:23:47.120 uh you can build state machines for
00:23:48.720 processes uh
00:23:51.400 management that uh can run without
00:23:54.320 weight without sleep or anything like
00:23:57.000 that uh you can
00:24:00.600 monitor parents from the child's
00:24:03.200 perspective perspective so when when you
00:24:05.360 fork you can get the parent ID and you
00:24:08.240 can just
00:24:09.880 uh create a pitf pointing back to your
00:24:13.520 parent so you make sure that your uh
00:24:16.559 worker is not
00:24:17.799 orphaned and if it is if it isn't uh
00:24:21.840 meaning if the parent is alive
00:24:23.440 everything is fine if it's not then well
00:24:25.600 most of the time you want to kill uh
00:24:27.919 such a such a node uh because you you
00:24:30.400 don't want to have uh processes running
00:24:32.480 without proper supervision
00:24:36.760 right there are some limitations to this
00:24:39.520 API so it's Linux 5.3 only but it's been
00:24:42.880 six years so come
00:24:45.039 Uh it's not available because it's Linux
00:24:47.679 only it's not available on Mac OS uh
00:24:50.640 surprise surprise uh FFI is pretty
00:24:53.919 stable i don't see it as a as a supply
00:24:56.840 chain issue
00:25:00.720 uh but it is limited to processes within
00:25:04.240 the same pit namespace and what that or
00:25:06.640 parent namespace uh which means that you
00:25:08.960 cannot use pit fd API to mon to monitor
00:25:13.120 for example processes from other docker
00:25:15.559 containers from the same docker
00:25:17.600 container yes from the host to the
00:25:19.440 docker container yes from one docker
00:25:21.840 container to the other no uh there are
00:25:25.200 no group operations
00:25:27.600 uh p api support uh sending signals to
00:25:31.600 groups with uh pitf you have to do it
00:25:34.400 yourself if you decide to use uh group
00:25:37.559 operations and there's no way to easily
00:25:41.200 access uh descendants like automatically
00:25:44.559 you have to build it yourself as well uh
00:25:47.440 so when to use it almost
00:25:51.080 never mean it like I yeah you know it's
00:25:54.480 the most professional API you can use
00:25:57.880 uh but it it tends to be an overkill uh
00:26:02.799 so what uh but it shines if you have uh
00:26:07.279 if you want to monitor nondirect child
00:26:10.240 processes it shines when you
00:26:13.400 uh want to use the most professional
00:26:16.039 APIs uh it shines when you have high
00:26:19.200 process churn when you have a lot of
00:26:21.200 forks uh that do shortlived stuff and so
00:26:24.240 on with like a lot of things happening
00:26:25.919 it may be better than standard pete
00:26:29.200 based uh
00:26:31.559 solution but for most of the longlived
00:26:35.799 uh swarm like systems where where things
00:26:40.159 are forked once and they keep on living
00:26:42.480 for days or longer it most likely
00:26:45.919 doesn't have any sense uh so you
00:26:50.880 probably don't want to use it and this
00:26:52.240 is what I learned uh when after I
00:26:54.799 released that to production uh but what
00:26:57.520 I also learned is that Linux has a lot
00:26:59.440 of weird and nice APIs that we don't uh
00:27:03.200 support out of the box in Ruby uh
00:27:06.559 process management is really hard like
00:27:08.720 much harder than I initially thought uh
00:27:11.840 KJ is awesome the rest of the core team
00:27:14.320 people are also great but KJ is like
00:27:16.760 yeah and
00:27:19.240 uh process referencing can be
00:27:22.360 stable but PD is a solution for edge
00:27:27.520 cases not something that should be used
00:27:30.720 always uh I don't think I'm going to
00:27:32.880 submit pull requests to Puma and other
00:27:35.039 solutions to uh to migrate to PDF uh but
00:27:39.279 I think I'm going to keep working on it
00:27:42.000 uh I'm going to extract this out of
00:27:44.080 Karafka to a standalone gem i will try
00:27:47.039 to provide some fallback capabilities
00:27:48.960 that would be bit based uh I do want to
00:27:52.320 look into real time i already started
00:27:54.399 actually looking into real time uh
00:27:56.200 signals uh to be able to have support
00:27:59.919 them from Ruby applications uh
00:28:03.120 especially the ones that use the seek Q
00:28:06.320 and I would love for Ruby to have uh
00:28:10.080 ability to read read uh real time
00:28:13.360 signals
00:28:14.960 uh
00:28:15.720 payloads uh during uh signal handling
00:28:19.440 because it could it can I see some use
00:28:21.679 cases for it especially when you have uh
00:28:24.480 for example a worker that you want to
00:28:26.799 shut down gracefully but within certain
00:28:28.880 time limitations you could send a signal
00:28:31.279 with a payload having the time
00:28:32.919 limitations and so on so uh that's
00:28:36.799 something I plan to work on uh this
00:28:40.360 year and yeah that that's all from me
00:28:44.159 for today so if you have any questions
00:28:47.120 I'm happy to answer them if not uh you
00:28:49.679 know after party or two thank you
Explore all talks recorded at RubyKaigi 2025
+66