Download a static file from a remote URL by using requests in python:

# author -> Roman Podlinov
 def download_file(url):
     local_filename = url.split('/')[-1]
     r = requests.get(url, stream=True)
     with open(local_filename, 'wb') as f:
         for chunk in r.iter_content(chunk_size=1024):
             if chunk: # filter out keep-alive new chunks
                 f.write(chunk)
     return local_filename

NOTE: stream=True parameter is used for returning large files in chunk and save it into a file

#python #requests #stream #large_file #download

Change stream is a new method on MongoDB 3.6 that you can use to watch real-time data modifications to get changes. In older versions you had to tail the oplog.

for change in db.collection.watch():
     print(change)

The ChangeStream iterable blocks until the next change document is returned or an error is raised. If the next() method encounters a network error when retrieving a batch from the server, it will automatically attempt to recreate the cursor such that no change events are missed. Any error encountered during the resume attempt indicates there may be an outage and will be raised.

#mongodb #mongo #mongo36 #change_stream #stream #etl