项目作者: mt592
项目描述 :
Big Data coursework using Hadoop/Spark, and concepts of mapreduce, graph theory, text mining, machine learning, etc.
高级语言: Jupyter Notebook
项目地址: git://github.com/mt592/big-data.git
Big Data Course
A compilation of work completed in the Big Data course I took during my master’s program.
Projects completed:
- acm_citation_analysis (final project) - using the weighted page rank algorithm, understand which papers in the ACM database get cited most frequently and are most ‘important’ in the network. Read more here.
- common_followers - explore which social media users mutually follow one another.
- common_movies - identify people with the highest overlap in common movie viewing.
- wikipedia_and_higgs - find symmetric wikipedia URLs). Create in-degree distribution of the higgs social media data set.
- yelp_average_stars - calculate the average stars per business on Yelp using MapReduce and Map-Combine-Reduce.
- yelp_restaurant_ml - Use text mining techniques and machine learning to predict the star rating of restaurants on Yelp using the raw text reviews.
Tech stack:
- Hadoop
- Scala
- Spark/PySpark
- Java