MS-DS Master of Data science FREE Master of Data science Big Data Questions and Answers 2 — Questions and Answers
Question 1: Which distributed computing framework uses Resilient Distributed Datasets (RDDs) as its core data abstraction?
- Apache Spark (Correct answer)
- Apache Hadoop
- Apache Flink
- Apache Storm
Correct answer: Apache Spark
Apache Spark uses RDDs as its fundamental data structure for fault-tolerant parallel processing.
Question 2: In the context of big data, what does the term 'data lake' primarily refer to?
- A centralized repository storing raw data in its native format (Correct answer)
- A relational database optimized for analytics
- A cloud-based data warehouse with predefined schemas
- A real-time streaming data pipeline
Correct answer: A centralized repository storing raw data in its native format
A data lake stores vast amounts of raw data in its original format until it is needed for analysis.
Question 3: What is the primary purpose of Apache Kafka in a big data architecture?
- Distributed event streaming and message brokering (Correct answer)
- Batch processing of large datasets
- In-memory caching of query results
- Schema management for data warehouses
Correct answer: Distributed event streaming and message brokering
Apache Kafka is designed as a distributed event streaming platform for high-throughput, real-time data pipelines.
Question 4: Which consistency model does the CAP theorem state cannot be fully achieved simultaneously with availability and partition tolerance?
- Strong consistency (Correct answer)
- Eventual consistency
- Causal consistency
- Read-your-writes consistency
Correct answer: Strong consistency
The CAP theorem states that a distributed system cannot simultaneously guarantee strong consistency, availability, and partition tolerance.
Question 5: What distinguishes columnar storage formats like Parquet from row-based storage in big data analytics?
- Columnar formats enable faster analytical queries by reading only relevant columns (Correct answer)
- Columnar formats store data in the order it was inserted for faster writes
- Columnar formats require less metadata than row-based formats
- Columnar formats cannot be compressed as efficiently as row-based formats
Correct answer: Columnar formats enable faster analytical queries by reading only relevant columns
Columnar storage reads only the columns needed for a query, dramatically reducing I/O for analytical workloads.
Question 6: In MapReduce, what operation occurs between the Map and Reduce phases?
- Shuffle and sort (Correct answer)
- Partition and replicate
- Compress and archive
- Index and cache
Correct answer: Shuffle and sort
The shuffle and sort phase transfers map output to reducers and sorts it by key so that identical keys are grouped together.
