CocoIndex is an ultra performant data transformation framework for AI, with incremental processing.

https://github.com/cocoindex-io/cocoindex

Simply connect to (Drive, S3, local files etc), write minimal code ~100 lines of python and ready for production. When sources get updates, it automatically syncs to targets with minimal computation needed.

We are ex-googler tech leads building data infra for years.