Data Engineer (2–6 Years)
About The Role The role designs, builds, and maintains the data pipelines and infrastructure that move data from raw sources to clean, analytical-ready tables clients rely on daily. You will work across the modern data stack - Python, Airflow, dbt, Snowflake, Kafka, and Spark - and will be expected to own your pipelines end-to-end.
Key Responsibilities
- Build and maintain ETL/ELT pipelines using Airflow or Prefect to move and transform data from operational databases, APIs, event streams, and SaaS sources
- Develop dbt data models following dimensional modeling and OBT patterns; implement dbt tests, documentation, and data lineage
- Administer and optimize Snowflake or Databricks environments: clustering keys, warehouse sizing, cost management, and query performance
- Build real-time streaming pipelines using Kafka and Spark Structured Streaming for event-driven data products
- Implement data quality monitoring using Great Expectations or dbt tests; build alerting pipelines for data SLA violations
- Collaborate with analytics engineers, data scientists, and ML engineers to ensure their data access and quality requirements are met
- Write clear technical documentation for pipeline architecture, data dictionaries, and on-call runbooks
What We Are Looking For
- 2–6 years of data engineering experience with evidence of production pipeline ownership
- Python proficiency: pandas, PySpark, and scripting for data processing and orchestration
- Airflow or Prefect for workflow orchestration; hands-on experience designing DAGs in production
- SQL expertise: complex queries, window functions, performance tuning across Snowflake, BigQuery, or Redshift
- dbt experience (any version): model development, testing, and documentation
- Familiarity with at least one cloud data platform: Snowflake, BigQuery, or Databricks
- Bonus: Kafka/Flink for streaming, Delta Lake, dbt Semantic Layer, or Fivetran/Airbyte connector management
Location San Francisco Bay Area (Hybrid)
- New York City
- Chicago
- Dallas
- Seattle
- Boston
- Remote considered