common problem: I need a bigger memory as my data does not fit and I get memory errors.
(bad) solution: buy a bigger machine
solutions (python) oa: chunking, lazy API (polars vs pandas), parquet instead of cvs,...
ref:
https://pythonspeed.com/articles/polars-memory-pandas/
common problem: I need a bigger memory as my data does not fit and I get memory errors.
(bad) solution: buy a bigger machine
solutions (python) oa: chunking, lazy API (polars vs pandas), parquet instead of cvs,...
ref:
https://pythonspeed.com/articles/polars-memory-pandas/