INR 1499/- for
What is the quality of data?
Is it good enough to be collected, consumed, and interpreted for business usage?
And how should we use this data?
Many more questions when a tester involves in testing applications with big data, AI, IoT, and analytical solutions.
Ambiguity has always been a key challenge for testers – be it with the ambiguous definition of requirements or with unstable test environments. But testing a data, big data workflow adds a completely new level of uncertainty to a tester’s life for modern technologies.
Data Validation is simply verifying the correctness of data. The Big Data Testing Pipeline consists of horizontal workflows where data transformations occur continuously, managing a series of steps that process and transform the data. The obtained result can be settled into a database for analysis(Machine Learning Models, BI reports or act as an input to other workflows.
This session is to provide a solution to challenges faced while data testing for an application (with big data, IoT, a mesh of devices, artificially intelligent algorithms and with data analytics, like:
The research employed an open-source solution for the implementation. Apache Kafka was used to gathering Batch data and Streaming data (Sensor/Logs). Apache Spark Streaming consumed the data from Kafka in Realtime and carried out the validations in Spark Engine. Further in the workflow, the data was stored in Apache Cassandra and then configured in Elasticsearch and Logstash to generate real-time reports/graphs in Kibana. The proposed tool is generic as well as highly configurable to incorporate any open-source tool you require to work within streaming, processing, or storing the data. The system includes configuration files where every single detail of the dependent tool used is appended and can be modified according to the needs.
This solution aims to analyze various Key performance indicators for Big Data like a data health check, downtime, time-to-market as well as throughput, and response time. The tool can be considered as a pluggable solution that can efficiently drive big data testing and uplift the data quality for further usage.
Benefits & Takeaways: Attend this session to understand the basic need of future application testing.