Description

In a dedicated endeavor to ensure data integrity within an organization, I spearheaded a data quality project that focused on reconciling discrepancies between two exports of the same database. The project's aim was to develop a systematic approach for weekly comparisons, presenting the findings through an insightful Power BI report. The initial phase involved extracting data from both databases using a Python notebook, a process that demanded not only technical acumen but also a strategic approach to handle large volumes of data efficiently.

With the data extraction process in place, I crafted a dynamic Power BI report that highlighted discrepancies between the datasets, offering clear visualizations of data mismatches and anomalies. The final, and perhaps most critical, component of the project was the automation of the data update process, set to refresh on a weekly cycle. This required meticulous planning to optimize performance, ensuring the timely and accurate retrieval of data. The automation process was fine-tuned to account for performance bottlenecks, resulting in a robust system that consistently delivered reliable data quality reports, empowering stakeholders to make informed decisions based on the highest quality data.