4 posts tagged with "Performance"

GeoETL v0.3.0 adds full GeoParquet format support with production-ready performance.

Key highlights:

• ✅ Full read/write support
• ⚡ 3,315 MB/min processing throughput
• 💾 Streaming architecture with minimal memory
• 🚀 Handles 100M+ features efficiently

TL;DR: We built systematic benchmarking for GeoETL to understand performance across formats. Results: CSV is production-ready (2.3 GB/min, 50 MB RAM), while GeoJSON needs optimization (297 MB/min, 84 MB RAM). Benchmarking revealed a 7.6x performance gap and guides our optimization roadmap. Here's how we built the infrastructure and what we learned.

TL;DR: GeoETL v0.2.0 delivers on our performance promises with a production-ready streaming architecture. CSV processing achieves 2,266 MB/min throughput (7.6x faster than GeoJSON) while maintaining constant O(1) memory usage. Process 4.2 GB files in just 49.9 MB of memory.

TL;DR: We're excited to announce the first release of GeoETL, a next-generation geospatial ETL tool built from the ground up with Rust, Apache DataFusion, and Apache Arrow. Our goal is ambitious: deliver 5-10x faster performance than GDAL while providing seamless scalability from your laptop to a distributed cluster.