项目作者: mt592

项目描述 :
Big Data coursework using Hadoop/Spark, and concepts of mapreduce, graph theory, text mining, machine learning, etc.
高级语言: Jupyter Notebook
项目地址: git://github.com/mt592/big-data.git
创建时间: 2020-06-22T01:26:28Z
项目社区:https://github.com/mt592/big-data

开源协议:

下载


Big Data Course

A compilation of work completed in the Big Data course I took during my master’s program.

Projects completed:

  • acm_citation_analysis (final project) - using the weighted page rank algorithm, understand which papers in the ACM database get cited most frequently and are most ‘important’ in the network. Read more here.
  • common_followers - explore which social media users mutually follow one another.
  • common_movies - identify people with the highest overlap in common movie viewing.
  • wikipedia_and_higgs - find symmetric wikipedia URLs). Create in-degree distribution of the higgs social media data set.
  • yelp_average_stars - calculate the average stars per business on Yelp using MapReduce and Map-Combine-Reduce.
  • yelp_restaurant_ml - Use text mining techniques and machine learning to predict the star rating of restaurants on Yelp using the raw text reviews.

Tech stack:

  • Hadoop
  • Scala
  • Spark/PySpark
  • Java