Sr. Big Data Developer

About the job

The Hands On Big Data Developer will work hand in glove with the development team to work on portions of our next generation data platform, pilot advanced data algorithms and data models, and leverage the Hadoop/Spark ecosystem in the cloud to combine very large data sets from retailers, market surveys, credit cards, GPS and more to deliver deep market analysis and predictions across multiple industries.


The right candidate will join a dynamic and skilled technology team and report to the SVP of Data Systems

Location: New York, Chicago, Texas, Toronto


  • Design, pilot Big Data & Analytics data models and algorithms with optimal performance and ease of maintenance.
  • Participate in the evaluation and pilot of new technologies and designs to support architecture components like data warehouse, analytics dashboard, APIs and more.
  • Participate in the design and implementation of an efficient, highly available, auto scaling cloud platform.
  • Assist and support Azure cloud migrations of existing data processes in classical infrastructure.
  • Coordinate efforts with IT and Operations to ensure that the Hadoop architecture, data structures and processes meet the business requirements and objectives.
  • Participate in new data projects working with business and product, and lead Proof of Concepts and pilots.
  • Provide technical thought leadership, bringing new ideas and approaches to the wider team.
  • Troubleshoot and benchmark existing systems to identify gaps and opportunities.
  • Assist in the ongoing development and documentation of the standards for the system and data processes.
  • Create project plans, manage milestones, create and distribute reports and manage risks
  • Communicate effectively with senior management, direct reports and customers.
  • Plan system capacity and design advanced features for future business growth and new business opportunities.


  • Passionate and pragmatic technologist that can apply technologies for effective solutions.
  • Proven experience with architecture design and hands-on implementation of Big Data pipe lines, preferably in Azure or AWS
  • Expertise with Hadoop, Spark and Hive implementations with strong programming experience in Hive, Java, Scala, Python, SQL.
  • Experience with both Relational and NoSQL databases, data modeling, database performance tuning.
  • Experience working on UNIX platform, expertise in shell scripting and performance monitoring on UNIX servers.
  • Experience with performance bench-marking and optimization.
  • Excellent analytical/problem solving and troubleshooting skills.
  • Excellent verbal and written communication, presentation and interpersonal skills
  • Bachelor or Master’s degree in Computer Science, Math, Physics or Engineering.
  • Experience with Machine Learning, Search Engines a plus
  • Experience with Analytics and Advanced Data Visualization a plus.

The NPD Group, Inc. is an Affirmative Action/Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or protected veteran status or any other characteristic protected by law.


Discover More AI Jobs:


More Information

Apply for this job Apply via Facebook
Share this job

We are one of the largest AI Communities online. Our publications have over 8.5 Million Views Annually and we have over 120K subscribers.