Allowing small allocations to fail

The general principle is to notify the caller. If the caller was a userspace program that made a system call, and the system call can't complete because there isn't enough memory, you return a failure status such as ENOMEM. If the caller was a kernel routine, again you return the failure status. In neither case is hanging indefinitely really a sensible thing to do - although certainly a limited amount of waiting and retrying can be a good idea.