Vaga de parceiro

Big Data Consultant - Campos dos Goytacazes / RJ

Disponível para Assinantes
Salvar nos Favoritos
  • Compartilhe:

Detalhes da Vaga

  • Escolaridade Não Informado
  • Segmento Não Informado
  • Salário Não Informado
  • Área de AtuaçãoDiversos / Outros

O que você irá fazer

  • As a Big Data Engineer, you will be responsible for designing, developing, and optimizing scalable data pipelines and architectures that enable data-driven decision-making across the organization.
  • You''ll work closely with data scientists, analysts, and software engineers to ensure reliable, efficient, and secure data infrastructure.
  • Key Responsibilities: Design, develop, and maintain robust and scalable data pipelines for batch and real-time processing.
  • Build and optimize data architectures to support advanced analytics and machine learning workloads.
  • Ingest data from various structured and unstructured sources using tools like Apache Kafka, Apache NiFi, or custom connectors.
  • Develop ETL/ELT processes using tools such as Spark, Hive, Flink, Airflow, or DBT.
  • Work with big data technologies such as Hadoop, Spark, HDFS, Hive, Presto, etc.
  • Implement data quality checks, validation processes, and monitoring systems.
  • Collaborate with data scientists and analysts to ensure data is accessible, accurate, and clean.
  • Manage and optimize data storage solutions including cloud-based data lakes (AWS S3, Azure Data Lake, Google Cloud Storage).
  • Implement and ensure compliance with data governance, privacy, and security best practices.
  • Evaluate and integrate new data tools and technologies to enhance platform capabilities.
  • Required Skills and Qualifications: Bachelor's or Masters degree in Computer Science, Engineering, Information Systems, or related field.
  • 3+ years of experience in data engineering or software engineering roles with a focus on big data.
  • Strong programming skills in Python, Scala, or Java.
  • Proficiency with big data processing frameworks such as Apache Spark, Hadoop, or Flink.
  • Experience with SQL and NoSQL databases (e.
  • g.
  • , PostgreSQL, Cassandra, MongoDB, HBase).
  • Hands-on experience with data pipeline orchestration tools like Apache Airflow, Luigi, or similar.
  • Familiarity with cloud data services (AWS, GCP, or Azure), particularly services like EMR, Databricks, BigQuery, Glue, etc.
  • Solid understanding of data modeling, data warehousing, and performance optimization.
  • Experience with CI/CD for data pipelines and infrastructure-as-code tools like Terraform or CloudFormation is a plus.
  • Preferred Qualifications: Experience working in agile development environments.
  • Familiarity with containerization tools like Docker and orchestration platforms like Kubernetes.
  • Knowledge of data privacy and regulatory compliance standards (e.
  • g.
  • , GDPR, HIPAA).
  • Experience with real-time data processing and streaming technologies (e.
  • g.
  • , Kafka Streams, Spark Streaming).
  • Why Join Us: Work with a modern data stack and cutting-edge technologies.
  • Be part of a data-driven culture in a fast-paced, innovative environment.
  • Collaborate with talented professionals from diverse backgrounds.

Informações Adicionais

  • Quantidade de Vagas 1
  • Jornada Não Informado
 

Envie esta vaga para um amigo!

 

Cadastro Inicial

Preencha os campos abaixo
  • Ao criar uma conta, você aceita nossa Política de Privacidade e nosso Termo de Adesão
 

Acesse sua Conta

Preencha os campos abaixo

ou acesse com seu provedor
 

Recuperação de Dados de Acesso

Informe seu e-mail cadastrado