Ask On Data

Ask On Data is an open source, GenAI-powered data engineering tool developed by Helical IT Solutions. It allows users to create, manage, and schedule data pipelines through a conversational chat interface using plain English, making data engineering tasks accessible to both technical and non-technical users. The platform generates Apache Spark jobs behind the scenes, translating natural language instructions into executable data pipelines.

Key Features

• Chat-Based Interface: Users interact with the platform by typing plain English commands, which the AI interprets and converts into data pipeline operations. There is no need to learn SQL, Python, or other programming languages to perform basic data engineering tasks.
• Data Preview and Validation: With each transformation instruction, users see a real-time preview of how the data changes, allowing them to verify results before committing. An undo functionality enables reverting changes when needed.
• Pipeline Scheduling: Users can create end-to-end data pipelines and schedule them to run at specified frequencies. Options include full load, incremental load, and truncate-and-load modes.
• Code Control for Advanced Users: For those who need more control, the platform also supports writing SQL, Python, and editing YAML configuration files, particularly useful for handling edge cases.
• Action History: A comprehensive history of all actions performed on data is maintained, allowing users to track the evolution of their workflows.
• Multiple Data Sources: Supports flat files, APIs, databases, data lakes, data warehouses, and log files as data sources.
• Managed Cloud Service: Ask On Data is available as a managed service, eliminating infrastructure and maintenance concerns. Self-hosted deployment options are also available.

Use Cases

• Data Migration: Move data between different storage systems, databases, or cloud platforms using natural language instructions.
• Data Transformation and Cleaning: Perform data integration operations such as unions, joins, cleaning, wrangling, and custom calculations through conversational commands.
• Data Warehouse and Data Lake Population: Load and transform data into data warehouses or data lakes without writing complex ETL scripts.
• Democratized Data Engineering: Enable business analysts, data scientists, and BI users to prepare their own data without waiting for data engineering resources.

Pricing

Ask On Data offers an open source self-hosted version that is free to use. A managed cloud free tier supports Excel and CSV files up to 5 MB with community support. The enterprise plan includes managed hosting, all database connectors, scheduling, and priority support. Custom pricing is available by contacting the team.

Limitations

The GitHub repository is still under preparation and not yet publicly available with source code. The tool is relatively new compared to established data engineering platforms, and its community and ecosystem are still developing. Advanced transformations may still require SQL or Python knowledge for complex edge cases.