Hadoop Designer

We are currently looking to hire a Big Data / SQL Developer. This position will be a key part of a team that delivers data integration projects in support of Compliance, Legal, and Regulatory business functions and processes, moving us forward with our multi-year enterprise data strategy.

Responsibilities:

• Perform development of data integrations on Hadoop
• Adhere to industry standards and methodologies for software development
• Follow and provide feedback to technical guidance on standards, design patterns, architectural patterns and frameworks for services and integrations
• Collaborate with analysts, modelers and subject matter experts in domain and data modeling
• Understand and support metadata management and information science practices that unlocks data as an asset for the firm and improves its usability through its lifecycle
• Maintain technical documentation that facilitate understanding, maintaining, reusing and refactoring of code

Requirements:

• 4-8 years experience in design and development of Python or Scala and Spark Application
• Experience with Distributed SQL Query Engine for Big Data like Apache Drill or Presto.
• Experience with storage formats like parquet files, OCR, No-SQL databases (MapR DB, HBASE etc.)
• Experience with developing on a major Hadoop ecosystem (Cloudra, MapR etc.)

Core Competencies :

• Knowledgeable of Enterprise Data Lake patterns
• Experience with development against relational database (ex: Microsoft SQL Server, PostgreSQL)
• Experience with DevOps toolchain that enables CI/CD pipeline (e.g., Git, TFS, Jenkins, TeamCity, Octopus, Puppet)
• Experience with creation of metadata management artifacts such as data dictionary, business glossary, data flow map, data lineage diagram or ontologies
• Experience with design and development of services with high performance and scalability
• Experience with design, implementation and adoption of container tech stack (e.g., Docker, Kubernetes)
• Experience with development against non-relational database (e.g., key-value store, columnar store, graph database)
• Strong problem-solving skills (e.g., root cause analysis) to debug or improve existing systems

Digital Intelligence Systems, LLC (DISYS) is an Equal Opportunity Employer that recruits and hires qualified candidates without regard to race, religion, sex, sexual orientation, gender identity, age, national origin, ancestry, citizenship, disability, or veteran status.