Senior Software Engineer (Apache NiFi & Node.js) Contract NEW

Role: Senior Software Engineer (Apache NiFi & Node.js)
Location: Dubai, United Arab Emirates
Engagement: Contractual
Duration: 12 Months (Extendable, Long-term with Marc Ellis)

Job Summary:

The Senior Software Engineer (Apache NiFi) will design, implement, and optimize secure, high-performance data flows using Apache NiFi, while also developing and maintaining Node.js backend services/APIs. The role requires strong hands-on experience building, managing, and tuning NiFi clusters, integrating with Kafka, MongoDB, and Cloudera CDP, and implementing secure data transfer and encryption. The candidate must bring a DevOps mindset, automation focus, and proven production NiFi experience.

Education:

• Degree, Postgraduate in Computer Science or related field

Key Responsibilities:

• Design, develop, and optimize complex NiFi data flows in production environments.
• Integrate NiFi with enterprise ecosystems such as Cloudera Data Platform (CDP) and Cloudera Manager.
• Migrate NiFi flows across major version upgrades, ensuring backward compatibility and minimal downtime.
• Develop and maintain backend services, integrations, and REST APIs using Node.js to support data pipelines and system integrations.
• Implement Groovy-based scripting in ExecuteScript and InvokeScriptedProcessor processors for custom logic.
• Configure and manage secure SFTP/SSH connections, including key-based authentication and permission handling.
• Ensure secure data transmission using NiFi’s encryption processors (e.g., EncryptContent) and proper key management.
• Develop and maintain integrations between NiFi and MongoDB using GetMongo, PutMongo, and QueryMongo processors.
• Build and manage Kafka-based pipelines, including Publish/Consume processors and schema management with Confluent Schema Registry.
• Tune JVM and resource configurations in RHEL environments for optimal NiFi performance.
• Implement monitoring, alerting, and metrics visualization using Prometheus/Grafana or Cloudera Manager.
• Deploy NiFi flows using NiFi Registry, manage version control, and maintain parameter contexts.
• Configure and maintain secure clustered NiFi environments with TLS certificates, LDAP/OIDC authentication, and access policies.
• Utilize NiFi debugging tools (Data Provenance, bulletins, logs) to resolve performance bottlenecks and flow errors.
• Participate in CI/CD automation for NiFi Registry deployments and ensure best practices for DevOps workflows.
• Support CI/CD workflows for NiFi Registry, including automated deployment and version promotion.
• Document NiFi dataflows, configurations, and best practices for operational and engineering teams.
• Collaborate effectively with cross-functional teams and provide guidance or mentoring when needed.

Technical Skills:

• 8+ years of overall software/data engineering experience, including a minimum of 4 years of hands-on experience with Apache NiFi in production environments.
• Proven experience with Cloudera NiFi (CDP DataFlow) and integration with Cloudera Manager.
• 5+ years of hands-on Node.js experience, including backend API development using JavaScript/TypeScript.
• Experience in building RESTful APIs or microservices using Express.js, NestJS, or similar frameworks.
• Strong proficiency in Groovy scripting for NiFi processors.
• Solid understanding of SSH/SFTP protocols, authentication schemes, and secure file transfer within NiFi.
• Expertise in data encryption, key management, and secure FlowFile handling.
• Experience integrating NiFi with MongoDB and Kafka ecosystems.
• Proficiency in NiFi record-based processing (Avro, JSON, CSV) using RecordReader/RecordWriter services.
• In-depth knowledge of controller services, including SSLContextService, DBCPConnectionPool, and Record services.
• Strong understanding of Zero-Master clustering model, node coordination, and site-to-site protocol.
• Experience debugging and optimizing dataflows using Data Provenance and performance tuning techniques.
• Strong understanding of RHEL administration, including systemd services, permissions, log rotation, and JVM tuning.
• Experience working in secure or regulated enterprise environments with strict audit/compliance controls.
• Exposure to custom NiFi processor development using Java or Groovy.