Data Engineer

Reporting to the Manager, BI Development, the Data Engineer will be responsible for integrating enterprise wide data from Cofense’s Cyber Security based Applications, Micro-services and other disparate data sources. Essential Duties/Responsibilities include designing Data Platform and Architecture, developing Data Pipelines on Cloud Technologies, and securing data end-to-end by complying data privacy rules.

Requirements

• Substantial experience with SQL and no-SQL OLTP databases and OLAP data warehousing technologies, especially AWS Aurora
• Experience data modeling and building data pipelines for multi-product and/or multi-department organizations
• Develop Data Pipelines on Cloud Technologies like Azure/AWS with well-defined tool frameworks
• Able to develop ETL code to stream data from disparate (structured and semi-structured) SaaS product data stores to Data Lake/Data Warehouse using Python, Azure/AWS Data Lake services
• Ability to write complex SQL scripts and automate them using Python
• Develop test cases and unit tests for key implementations of Data Platform by adhering to software engineering best practices and standards
• Secure data end-to-end by complying data privacy rules while developing processes to move data across Applications/Data Sources and Data Lake/Data Warehouse, as well as while delivering data through SQL clients and BI tools.
• Experience building adhoc reporting APIs and service layer on top of underlying OLTP and OLAP databases
• Help integrate Data Platform with BI tools like Power BI, Tableau, Splunk etc.
• Ability to develop and interpret Entity Relationship Diagrams (ERD) across data sets in relational database systems as well as non-relational Data stores
• Able to do Data Mining and Identify trends, patterns, anomalies in complex data sets across multiple data sources/systems and present results without ambiguity
• Develop data transformations to generate Facts, Summaries, Key metrics by applying business rulesets and aggregations using Python, SQL and other transformation tools
• Able to review current processes related to data ingestion, transformation and statistical analysis and re-engineer them
• Collaborate with business users across Cofense’s departments in defining requirements, prioritize project work and deliver them timely
• Expertise in SQL skills for data transformations, statistical analysis, and troubleshooting across more than one Database Platforms (MySQL, PostgreSQL, Redshift, Azure SQL Warehouse etc.)
• Expert in writing complex SQL scripts and automate them using Python
• Knowledge of Data management on NoSQL DBs like DynamoDB, Mongo, and know-how of Big Data tools Hadoop, Spark, Kafka/Kinesis/SQS/Azure Queues or other messaging tools is huge plus
• Analytical skills, with good at finding data trends/outliers, anomalies, and articulate complex information or data points with Business Users, Management, and individuals
• Enthusiasm to work with lot of data across disparate data sources and Databases
• Has strong sense of engineering craftsmanship, takes pride in the code they write.
• Has a sense of intellectual curiosity and a burning desire to learn is self-driven, actively looks for ways to contribute, and knows how to get things done
• Is deliriously customer-focused both internal and external customers
• Sees big picture impact and relationships among and across work units
• Identifies complex technical problems and tries to resolve with minimal help

Benefits

• 401k Matching
• Retirement Plan
• Tuition Reimbursement