A realtime developer's checklist

Keep in mind that even if *your* real-time application pre-allocates and locks all memory, the *kernel* code you invoke via system calls isn't written with the same discipline in mind. Kernel mutexes, for example, are not real-time mutexes by default, so you can still get into priority inversion situations with the rest of the system by invoking kernel code that takes locks owned by non-RT tasks -- for example, the task list lock or an inode lock. (The RT patchset is meant to mitigate this problem.)

On a vanilla kernel, you can use *some* OS services without risking priority inversion, but you have to be careful about which ones they are and audit the kernel-side code paths you invoke and make sure that one of your fancy important SCHED_FIFO threads isn't going to get blocked behind some slowpoke non-RT part of the system because the SCHED_FIFO thread made a system call that takes a lock while it's running in the kernel.

If I had my way, we'd just apply the RT patchset universally, *and* make pthreads in gblic use PI by default, *and* implement surrogate execution so that priority inheritance did something for SCHED_OTHER threads (i.e. normal non-RT processes). (Right now, priority inheritance is useless for SCHED_OTHER.) I'm not likely to get my way because my proposal would tank throughput on batch workloads, but personally, I value consistency of performance over throughput.