Assignment 2 — Spark

Usage

Run the following command in the terminal:

spark-submit Assignment2.py

This will generate a text file, ass2-eda-result.txt, in the same folder as Assignment2.py.

Output

The result file contains the class proportion for the instances, in the same order in which the instances appear in ass2-eda-18.txt.

A sample ass2-eda-result.txt (generated from a prior run) is included in this repository.

Requirements

• Apache Spark (with spark-submit available on the PATH)

Files

File	Description
Assignment2.py	Spark script that computes class proportions
ass2-eda-18.txt	Input data file
ass2-eda-result.txt	Output file with class proportions per instance