Allowing small allocations to fail [LWN.net]

Allowing small allocations to fail

Posted Mar 11, 2015 10:24 UTC (Wed) by andresfreund (subscriber, #69562)
Parent article: Allowing small allocations to fail

> Peter Zijlstra observed that ENOMEM is a valid return from a system call.

FWIW, that's not generally true. E.g. close() is not documented to return ENOMEM and afaics could end up returning ENOMEM under the new regimen (close => __close_fd => filp_close => f_op->flush).

to post comments

Allowing small allocations to fail

Posted Mar 11, 2015 12:22 UTC (Wed) by mathstuf (subscriber, #69389) [Link] (4 responses)

Close returning failure already has confusing semantics. Is the FD closed? So far, Linux won't leave it open while HP-UX will. There was an article on it here not too long ago and I think a bug was filed for POSIX to specify it as "must be closed".

Allowing small allocations to fail

Posted Mar 11, 2015 12:37 UTC (Wed) by andresfreund (subscriber, #69562) [Link]

close() was just the first example that came to mind. There's many more. E.g. fsync(), which certainly looks like it could return ENOMEM after the change given the variety of things done on its behalf in various FSs.

Allowing small allocations to fail

Posted Mar 11, 2015 18:24 UTC (Wed) by gutschke (subscriber, #27910) [Link] (1 responses)

We ran into that problem a while ago. I had gotten into the habit of checking return codes for all system calls, and close() was the one that would occasionally fail for no discernible reason, but still (sometimes?) close the file descriptor.

This makes error handling super difficult -- especially if you need to worry about other threads opening new file handles concurrently. It is then impossible to accurately tell whether the original file descriptor is still open or whether it has been replaced by a completely unrelated one.

I don't recall if we ever managed to pin-point the root cause. It might have only happened with some kernel versions, or with specific file systems (or maybe with non-standard user-space wrappers around the system call). I don't think we ever actually managed to reproduce the failure in house.

If I recall correctly, we ran into particularly nasty errors for a while, as we had wrapped close() into TEMP_FAILURE_RETRY(), which apparently is just not a safe thing to do.

My conclusion was that for close() I should not check error codes, and hope for the best, as I can neither trust the error code nor is there really any way I can handle the error without possibly making things worse. This is particularly frustrating, as the manual page explicit warns against ignoring error codes...

Allowing small allocations to fail

Posted Mar 12, 2015 23:39 UTC (Thu) by peter-b (guest, #66996) [Link]

It's not possible to handle close(2) errors on Linux. The file descriptor is deallocated *before* any condition that could cause failure can be hit -- so no matter what happens, the file descriptor is invalid when close(2) returns and you can't do anything else with it.

Allowing small allocations to fail

Posted Mar 11, 2015 21:25 UTC (Wed) by wahern (subscriber, #37304) [Link]

http://austingroupbugs.net/view.php?id=529

The resolution is that close will never fail. HP-UX was the odd man out, and it seems that they'll just have to suck it up, although I doubt HP-UX is even compliant to the current POSIX revision, let alone the upcoming one.

Allowing small allocations to fail

Posted Mar 14, 2015 12:48 UTC (Sat) by epa (subscriber, #39769) [Link]

If close() is not documented to return ENOMEM, then the kernel needs to make sure that out-of-memory conditions cannot occur on close(), by reserving enough memory in advance (perhaps on open()). Having it hang forever instead is not really an answer, since close() is not documented to hang either.