2

I have a class I use to track the progress of an ftp download (using ftplib).
I use ftplib with multiprocessing to download several files at once.
This is my first attempt at using a class in Python and I am doubtful whether I am closing a file I opened inside the class. The code snippets I think relevent are given below:

class track_transfer:
    def __init__(self, size, fname):
        self.file_name=fname
        self.size_in_bytes=size
        self.downloaded=0
        self.file=open(fname, 'wb')

    def __del__(self):
        try:
            self.file.close()
            self.file=None
        except:
            pass

    def update(self):
        self.downloaded=self.file.tell()
        if self.downloaded==self.size_in_bytes:
            self.file.close()

As you can see, I try to close the files at two places: one inside the finalizer another in a class method update().
What I wanted to know is will my file be closed correctly when I use this class?
Or is there a better method to follow?

UPDATE
I think I may need to provide more information here.

The class track_transfer has another method named progress(self, data) defined as follows:

    def progress(self, data):
        self.file.write(data)
        self.update()

Instantiating the classAdded more information,

tracker=track_transfer(size=fsize, fname=fname)

and downloading the file,

ftp.retrbinary("RETR %s" %(fname),
        callback=tracker.progress)

where ftp.retrbinary() is a method defined in the ftplib library to retrieve a file as binary from an ftp server. This will call the tracker objetc's progress method whenever a block of data is received. This means that the method progress() will be called several times when downloading a file.

11
  • Python doesn't really have destructors. You must be thinking of C++.
    – quamrana
    Commented Oct 18, 2019 at 9:07
  • 3
    There's no point in your __del__ function. Python will close a file when the associated file object is garbage collected anyway: stackoverflow.com/questions/49512990/…
    – freakish
    Commented Oct 18, 2019 at 9:09
  • @quamrana I thought del in python is similar to destructors in C++? What are del called in python then?
    – RogUE
    Commented Oct 18, 2019 at 9:20
  • The __del__ method is called a finalizer.
    – quamrana
    Commented Oct 18, 2019 at 9:28
  • @freakish Then there's no point in using file.close() explicitly anywhere in a script, is there?
    – RogUE
    Commented Oct 18, 2019 at 9:29

1 Answer 1

2

The __del__ probably does not help: it will only be called when the track_transfer instance is garbage-collected, and it will presumably be the only thing that has a reference to the file - so the file would be garbage-collected anyway, and Python will close the file when that object is garbage-collected. (__del__ is for very special-purpose cases that you will probably not run into; Python is naturally a garbage-collected language, so we generally don't bother with anything like "destructors".)

The normal way to manage the lifetime of a file object is using a with block:

with open(path, mode) as f:
    # do file operations
# file will be closed when the block is exited, even if by an exception etc.

This works because a file is its own "context manager". Storing the file as a class attribute doesn't mesh too well with this approach, because you want to open the file in one class method call and close it at some later point. But we can fix this by making our class be a context manager, and using its context management instead of the file's.

7
  • Actually, the data is written to the file in a separate method of the class, which will be called repeatedly. That method then call the update() method. So, I am not sure whether this would work.
    – RogUE
    Commented Oct 18, 2019 at 9:38
  • @RogUE: So the track_transfer class does more than track the transfer? Maybe your design is the issue.
    – martineau
    Commented Oct 18, 2019 at 9:57
  • From my quick read of the ftplib docs, it seems that the download calls will all just block and wait for the whole download anyway - so you'd have to do some kind of threading thing to monitor the download, and there is no need to make repeated calls to get all the data. Commented Oct 18, 2019 at 9:58
  • From the source code doc 'callback: A single parameter callable to be called on each block of data read'. So the function progress will be called several times in a download which gives me an opportunity to update downloaded variable of the class.
    – RogUE
    Commented Oct 18, 2019 at 10:27
  • It's not mentioned in the docs, I got it from the source code at github
    – RogUE
    Commented Oct 18, 2019 at 10:29

Not the answer you're looking for? Browse other questions tagged or ask your own question.