Big Data Engineer Lead (Systems)
The NPD Group

Port Washington, New York

Posted in Science and Research


This job has expired.

Job Info


Become a part of our diverse team

NPD is an inclusive and innovative work environment where you can learn and work on meaningful initiatives with some of the best and brightest in the market research industry. The NPD Group provides the world's most successful brands with leading market research, combining consumer and retail point-of-sale data with analytic solutions to interpret today's market trends while anticipating tomorrow's.

Position Overview

Duties: Gather requirements to define data definitions, transformation logic, and data model logical and physical designs, data flow and process. Design, develop, and test data processes involving multiple platforms, including Big Data and Ab Initio. Perform data analysis using the tools Ab Initio, Python and Hive and solve any data related queries or issues. Develop, test and automate metadata upload/download, data quality control applications, release procedures using Ab Initio, Oracle SQL and PL/SQL, Unix Shell Scripts, and Hadoop Big Data Technologies. Identify bottlenecks and fine tuning Big Data Processes. Work independently in the development, testing, implementation and maintenance of systems of moderate-to-large size and complexity. Work closely with Quality Assurance, Operations and Production support group to devise the test plans and solve any data or processing issues. Document the data mapping, transformation logic and process flow for each development project. Create Enterprise Metadata repository and related web interface to store and deliver metadata information. Work closely with system users to solve any issues in data or system. Designing and implementing product features in collaboration with business and IT stakeholders. Design reusable Big data components, frameworks and libraries using Pig, Hive, Spark and HBase. Work closely with the Architecture group and driving solutions. Implement the data management framework for the Data Lake. Support implementation and driving to stable state in production. Provide alternative design solutions and project estimates.

Requirements: Master's degree in Computer Science or Computer Engineering and three years of experience in position offered or as Programmer/Analyst or Bachelor's degree in Computer Science or Computer Engineering and five years of post-Bachelor's progressive experience in position offered or as Programmer/Analyst. Three years of required experience must have included using Ab Initio, Hadoop, Unix Shell Scripting, Python, Oracle, PL/SQL, Hive, Spark, Pig and HBase.

Contact: Janice Ippolito, 917-991-4137 or Janice.Ippolito@npd.com.

The NPD Group, Inc. is an Affirmative Action/Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or protected veteran status or any other characteristic protected by law.


This job has expired.

More Science and Research jobs


Circana
Chicago, Illinois
$84,000.00 - $135,000.00 per year
Posted about 1 hour ago

Circana
Waterloo, New York
$15.00 per hour
Posted about 1 hour ago

Circana
Chicago, Illinois
$47,000.00 - $75,000.00 per year
Posted about 1 hour ago

Get Hired Faster

Subscribe to job alerts and upload your resume!

*By registering with our site, you agree to our
Terms and Privacy Policy.