Big Data Software Engineer
Complex problems require the right expertise! Today, one of the biggest challenges lies in data storage and processing, across these three main domains (3Vs): Volume, Velocity and Variety.
At Xpand IT, the Big Data technological area develops and implements architectures and software solutions that represent the state of the art on capturing, ingesting, storing and managing critical data from huge clusters where the 3Vs are always present. As concerns technology stack, we take advantage of almost every state-of-the-art framework in Big Data ecosystem such as Spark, Kafka, Hive/Impala, Azure Data Services or MongoDB using Java and Scala as programming languages to interact with them.
As a Big Data Software Engineer, you’ll play a key role in the team along with Big Data System Engineers to create the best software solutions, applying big data best practices in terms of architecture and systems design. You’ll ensure that backend software performance, reliability and scalability are achieved regardless of how fast data clusters grow or the throughput of information increases.
Your daily activities will include:
- Implementing software solutions that include loading, pre-processing and ingestion from disparate data sets into big data clusters.
- Designing robust and reliable software solutions regardless of the amount of input data.
- Implementing big data industry standards into the end software solution
- Analysing complex problems that need solutions for efficiently manipulating huge amounts of information
- Developing data models and data organisations that optimise “search” and “store” operations in large-scale distributed systems
- Designing APIs for swift data tracking and query data at high speeds
// Stacks: Java; Scala; HDFS, Hive, Impala, Spark, Kafka/Confluent; MongoDB, HBase
SKILLS YOU NEED TO HAVE
- MSc / BSc in Information Systems and Computer Engineering and/or Computer Science
- Sound knowledge of Linux operating systems
- Experience with Java-oriented technologies (JBoss, Spring, Spring MVC, Hibernate, REST/SOAP)
- Experience with design patterns to implement advanced software solutions.
- Experience with non-relational & relational databases (SQL, MySQL, NoSQL, Hadoop, MongoDB, etc.)
- Sound experience in using Eclipse, Maven, Jenkins, GIT, JIRAor equivalent tools
- Sound communication skills (written and spoken)
// Will be a nice plus if you have:
- Knowledge or experience with Apache Spark, Kafka or the Hadoop ecosystem
- Team player and problem-solving skills
// Learn more about Big Data area:
Eclipse, Maven, Jenkins, GIT, JIRA
XML, JSON, SOAP
André Pires, Big Data Software Engineer
A Big Data Software Engineer is challenged every day to build scalable and resilient data processing systems that are the groundwork for data-driven decisions.