Rubem Ribeiro de Barros

Architected, implemented, and orchestrated scalable data pipelines using EMR (Spark/PySpark/Scala), Airflow, and Lambda, integrating diverse data sources like TXT, CSV, Parquet, Iceberg, mainframe, SQL databases, and APIs.

Qualified, prioritized, and classified sensitive data, strengthening data governance and ensuring LGPD/PII compliance across massive datasets (billions of records).

Standardized and enriched data through schema normalization, consistent naming, data typing, deduplication, cleansing, and masking, reducing duplicates by ~15% and increasing critical field completeness to ~98%.

Stored and exposed data within the Silver layer using EMR/Scala with Glue Data Catalog, enabling efficient querying via Athena for multiple business units.

Automated deployment and continuous monitoring with Jenkins and Airflow DAGs, ensuring high availability and meeting critical SLAs.

Collaborated with Agile Scrum squads and business stakeholders to define and prioritize data engineering requirements.

Engineered and optimized data ingestion and transformation pipelines using Hadoop, Spark, Hive, Impala, and SQL, processing millions of daily fiscal records from diverse sources (databases, XML, TXT).

Implemented robust data quality checks, typing, and calculated fields before publishing, ensuring trusted datasets for downstream consumption by auditors and BI analysts.

Contributed to strategic fiscal modernization projects (NF3-e and DIMP), centralizing financial transaction monitoring across the state.

Provided trusted data for thousands of monthly queries, improving decision accuracy and increasing fiscal data audit speed by 90%.

Supported Pernambuco's largest fiscal modernization initiatives, impacting thousands of taxpayers and enhancing revenue effectiveness.

Developed and optimized ETL/ELT pipelines to ingest data from Mirakl sales/marketing APIs and Google Analytics 4 (GA4), consolidating into Cloudera CDP data warehouses and data lakes.

Created multidimensional data models to support KPI analysis for sales performance, marketing campaign results, user registration, and customer engagement.

Calculated sales and marketing KPIs for executive dashboards, reducing analysis time by 30% and speeding up decision-making.

Participated in the first Oi squad to work natively on Cloudera CDP, developing all pipelines and ensuring >95% reliability.

Processed millions of daily sales and access records in a big data environment (Hadoop, Hive, Impala, Spark, PySpark).

Collaborated in an Agile Scrum environment, ensuring continuous delivery and strong alignment with business and technical teams.

Produced weekly executive reports tracking KPIs for thousands of SKUs (e.g., smartphones, appliances, air conditioners).

Received leadership recognition (Distinctive Achievement) for ensuring delivery continuity during team transitions, taking on increased technical responsibilities and managing a team member.

Customized and maintained Oracle BRM 12.0, developing and tuning MTAs and Opcodes (C), pipeline configuration, and data modeling via PODLs and NAPs to ensure billing rule adherence.

Automated billing and revenue routines using Shell Script (Unix), SQL, and PL/SQL, creating batch jobs and deployments via Azure DevOps.

Generated and maintained invoices and reports with Oracle BI Publisher, including creation/updates of RTF templates and standardized layouts.

Provided integration and support for bill run cycles, troubleshooting incidents using logs and database queries, bug fixing, and continuous performance improvements.

Managed source control with Git and delivered solutions within Agile Scrum teams.