Empowering Businesses Through Digital Excellence

Job Description:

Data Engineering Architect [AWS]

Welcome to Awign Expert, a division of Awign - India's largest work-as-a-service platform. At Awign Expert, we connect skilled professionals with exciting contractual or project-based work opportunities offered by top companies. Our mission is to empower professionals by matching them with projects that align with their skills, interests, and experience.

At Awign Expert, we understand the challenges faced by independent professionals in finding meaningful work and managing administrative tasks. That's why we act as a dedicated HR office for our Experts, handling the entire onboarding process, providing continuous feedback, resolving conflicts, and ensuring seamless payroll management.

Our goal is to create a hassle-free environment for our Experts, allowing them to focus solely on their work without the burden of administrative complexities. By partnering with Awign Expert, professionals gain access to a vast network of renowned companies and projects across various industries.

About the Role:

• Duration: 6 months
• Location: Bengaluru
• Timings: Full Time (As per company timings)
• Notice Period: Immediate Joiner - Only
• Experience: 8–9 Years

JD Overview:

We are seeking a highly experienced Data Architect to design and implement robust, scalable, and high-performance data systems using modern technologies. The ideal candidate should have deep expertise in database architecture, distributed systems, data modeling, and hands-on proficiency in Python or Golang. You will be responsible for defining the architecture for structured, semi-structured, and vector-based data platforms across AWS infrastructure.

Mandatory Skills:

• PostgreSQL
• Redis
• Apache Iceberg
• Graph/Vector Databases
• AWS Cloud Platform (S3, EC2, RDS, Lambda, EKS, etc.)
• Languages: Python or Golang

Responsibilities:

• Data Architecture & Modeling:
  • Design normalized and denormalized data models for transactional and analytical workloads.
  • Define strategies for schema evolution and data versioning using Apache Iceberg or similar.
• Database Systems Expertise:
  • Architect solutions using PostgreSQL for OLTP and analytical use cases.
  • Implement Redis for low-latency caching, real-time analytics, and streaming pipelines.
  • Design high-performance access patterns for relational, key-value, and time-series data.
• Vector and Graph Data Architecture:
  • Integrate and manage graph/vector databases for AI/ML and recommendation engines.
  • Optimize vector search performance using indexes and hybrid search strategies.
• Cloud Infrastructure (AWS):
  • Deploy and scale data systems using AWS services like S3, EC2, RDS, Lambda, and EKS.
  • Ensure high availability, durability, and disaster recovery of data infrastructure.
• Data Governance & Security:
  • Implement best practices for data quality, lineage, privacy, and role-based access.
  • Establish backup, archival, and retention strategies for critical datasets.
• Programming & Scripting:
  • Develop data ingestion, transformation, and orchestration workflows in Python or Golang.
  • Write reusable modules and APIs for data access, quality checks, and job orchestration.

Requirements:

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related technical field.
• 8–10 years of industry experience in data engineering, architecture, or backend system design.
• Proven experience designing and implementing data platforms at scale on cloud infrastructure (preferably AWS).
• Expertise in PostgreSQL: schema design, indexing, performance tuning.
• Experience with Redis for caching, session stores, or message queues.
• Practical experience with Apache Iceberg for large-scale data storage and schema evolution.
• Understanding of vector databases (e.g., FAISS, Weaviate, Pinecone).
• Experience with graph databases (e.g., Neo4j, Amazon Neptune).
• Proficient in Python or Golang for data ingestion, scripting, and automation.
• Familiarity with CI/CD pipelines, monitoring, and infrastructure-as-code (e.g., Terraform, CloudFormation).
• Ability to design batch & streaming pipelines using tools like Apache Kafka, Airflow, or AWS Glue.
• Implement encryption, data masking, and access control.
• Familiarity with GDPR, HIPAA, or similar data compliance standards.
• Proficient in query optimization, data partitioning, indexing strategies, and caching mechanisms.