Software Engineer - Hadoop/Spark

Job Description:
Job Profile Summary
Designs, develops, troubleshoots and analyzes moderately complex software programs for computer based systems. Customizes software based upon research.

Skills required:
Experienced big data/Hadoop and Spark developer, with a strong background in file distributed systems.
Programming languages: Python, Scala, PySpark package
Scripting: Shell scripting
Hadoop ecosystem: Hive, Oozie, Spark Scala, Spark Python, Airflow
ETL: Talend

Job Description
Essential Job Functions

Performs moderately complex systems modeling, simulation and analysis to provide appropriate software solutions.
Develops software system testing procedures, programming and documentation to ensure standard use of procedures.
Provides input for documentation of new or existing programs to ensure information accessibility, as required.
Designs, develops, troubleshoots and analyzes software programs to ensure compatibility with hardware systems.
Customizes software based on research and customer needs as appropriate.
Coordinates and facilitates communications with other departments and/or commercial vendor(s) to investigate and resolve software matters of significance and to ensure proper functioning of systems.
Maintains communication with management and customers regarding status of software development and problems.
Acts as technical lead on large projects involving a particular software package(s).
Provides the customer base with first level support.

Basic Qualifications

Bachelor's degree or equivalent combination of education and experience
Bachelor's degree in computer science, management information systems, or related field preferred
Five or more years of experience in software engineering or operating systems programming
Experience working with design and coding languages
Experience working with software design principles
Experience working with computer hardware

Other Qualifications

Good analytical and creative problem solving skills for design, creation and testing of software
Good interpersonal skills to interact with customers and team members
Good communication skills to interact with team members and senior support personnel
Ability to work independently and as part of a team

Work Environment