Author Archives: Wes McKinney and Hadley Wickham

Feather: A Fast On-Disk Format for Data Frames for R and Python, powered by Apache Arrow

Categories: Data Science

This past January, we (Hadley and Wes) met and discussed some of the systems challenges facing the Python and R open source communities. In particular, we wanted to explore opportunities to collaborate on tools for improving interoperability between Python, R, and external compute and storage systems.

One thing that struck us was that, while R’s data frames and Python’s pandas data frames utilize different internal memory representations, the semantics of their user data types are mostly the same.

Read more