Hey data engineers! If you work with Parquet files regularly, you know how frustrating it can be to switch between different utilities and scripts. That’s why I’m excited to share a CLI tool I’ve been building, called nail-parquet, which aims to simplify Parquet file operations.
As someone who’s worked with Parquet files, I’ve encountered my fair share of pain points. That’s why I decided to create a tool that can handle various tasks, from basic data inspection to advanced analysis. The tool is built in Rust using Apache Arrow and DataFusion, making it fast and efficient for large datasets.
Currently, nail-parquet offers over 30 commands, including data manipulation, quality checks, file operations, and analysis tools. Some examples include filtering, sorting, sampling, deduplication, outlier detection, and frequency analysis.
As the project has grown, I’ve realized that there are still many areas that can be improved. That’s where you come in! If you have experience working with Parquet files, I’d love to hear about your pain points, workflows that could be streamlined, and features that would make your life easier.
The tool is open-source and available for installation via `cargo install nail-parquet`. I’d appreciate any feedback, ideas for new features, or suggestions for improvement. You can check out the repository on GitHub.
Let’s work together to create a tool that makes working with Parquet files a breeze!