daffodil, data frames for optimized data inspection and logical processing with Ray Lutz

data data frame

register

2025.03.05

Speaker: Ray Lutz

Ray Lutz

daffodil (data frames for optimized data inspection and logical (processing)), which can create data frame instances similar to pandas, but using conventional python data types.

This means no conversion to/from the Pandas world, which I have found from testing has a very high overhead. In fact, unless you plan to do at least 30 repetitive column-based operations (like sums, etc) then you should just stay in python world and avoid the conversion time, and you win. But for many, time is not of the essence, or they stay in Pandas world and never need any python. The syntax is easy to use and I am extending it to use SQL database to allow for large table size and use of the robust joins, etc. The SQL part is under work and not released yet.

Author

Gabor Szabo (szabgab)

Gabor Szabo, the author of the Python Maven web site.

Gabor Szabo