Using groupby in pandas in multi-thread fashion
Sometimes you just need to use all of your CPUs to process some nasty thing in pandas (because you are lazy to do it properly) quick and dirty.
Pandas'
Solution I googled
- https://gist.github.com/tejaslodaya/562a8f71dc62264a04572770375f4bba
My lazy way using tqdm +
- https://gist.github.com/snakers4/b246de548669543dc3b5dbb49d4c2f0c
(Savva, if you read this, I know that your version is better, you can also send it to me to share xD)
#ds
Sometimes you just need to use all of your CPUs to process some nasty thing in pandas (because you are lazy to do it properly) quick and dirty.
Pandas'
GroupBy: Split, Apply, Combine seems to have been built exactly for that, but there is also a lazy workaround.Solution I googled
- https://gist.github.com/tejaslodaya/562a8f71dc62264a04572770375f4bba
My lazy way using tqdm +
Pool- https://gist.github.com/snakers4/b246de548669543dc3b5dbb49d4c2f0c
(Savva, if you read this, I know that your version is better, you can also send it to me to share xD)
#ds
Gist
pandas DataFrame apply multiprocessing
pandas DataFrame apply multiprocessing. GitHub Gist: instantly share code, notes, and snippets.