Explicit way to close file in Python

Question

Please look at the following code:

for i in xrange(1,5000):
    with open(r'test\test%s.txt' % i, 'w') as qq:
        qq.write('aa'*3000)

It seems to be written according to all Python rules; files are closing after using. Seems to. But in fact it seems to recommend(!) system to close file, not to close it explicitly because when I'm looking on Resource monitor it shows a lot of open files . It gives me a lot of problems because in my script I use a lot of files and after a long time I got "Too many open files" error despite of 'closing' it from source code.

Is there some way to explicitly close file in Python? Or how can I check whether the file was really(!) closed or not?

Update: I've just tried with another monitoring tool - Handle from Sysinternals and it shows all correct and I trust it. So, it may be problem in Resource monitor itself.

Screenshot which shows files opened:

Answer 1

Your code

for i in xrange(1, 5000):
    with open(r'test\test%s.txt' % i, 'w') as qq:
        qq.write('aa' * 3000)

is semantically exactly equivalent to

for i in xrange(1, 5000):
    qq = open(r'test\test%s.txt' % i, 'w')
    try:
        qq.write('aa' * 3000)
    finally:
        qq.close()

as using with with files is a way to ensure that the file is closed immediately after the with block is left.

So your problem must be somewhere else.

Maybe the version of the Python environment in use has a bug where fclose() isn't called due to some reason.

But you might try something like

try:
    qq.write('aa' * 3000)
finally:
    # qq.close()
    os.close(qq.fileno())

which does the system call directly.

Answer 2

You should be able explicitly close the file by calling qq.close(). Also, python does not close a file right when it is done with it, similar to how it handles its garbage collection. You may need to look into how to get python to release all of its unused file descriptors. If it is similar to how it handles unused variable then it will tell the os that it is still using them, whether or not they are currently in use by your program.