ECN-AlexNet-Tensorflow-PROSAGA-码农传奇

AlexNet

Tensorflow 2.x Implementation of the original AlexNet Paper

Paper Link
·
Architecture
·
Data Loader
·
Model Trainer

AlexNet Architecture

About The Project

AlexNet is a Deep Learning Paper published in the year 2012 by Alex Krizhevsky (Hence, the name). It is one of the pioneer Deep Learning Publications which kick started the Deep Learning Research and proved its importance.

This implementation is a part of my learning where I take an attempt to implement Key Deep Learning Papers using Tensorflow or PyTorch.

The Original Literature Can be found here: AlexNet Paper

Paper Explained Here: ImageNet Classification with Deep Convolutional Neural Networks - YouTube

Quick Links

The Original Literature Can be found here: AlexNet Paper
Paper Explained Here: ImageNet Classification with Deep Convolutional Neural Networks - YouTube
Data Loader: GitHub | nbViewer
Model Architecture: GitHub | nbViewer
Model Trainer: GitHub | nbViewer
Trained Weights: Epoch 35, Categorical Accuracy 0.48

Training/Validation Resources

Trained Weights

The final Trained Weights after automatic early stopping can be found here: Epoch 35, Categorical Accuracy 0.48

Legend

Orange = Training Curves
Blue = Validation Curves
Training Curves

Features

Tf.Data - Optimized Tensorflow Data Pipelining using Tf.Data and Tensorflow Datasets
Vanilla Implementation - I have taken an attempt to implement the original AlexNet Architecture as resembling the original literature as closely as possible.
Fused Implementation - Since the Original Implementation was not very efficient for a single GPU training, hence, I have also implementation a Fused Model which closely resembles the original, but is a single straight network designed to run on a single GPU.
Mixed Precision Training - The Implementation uses FP16 - Mixed Precision Training since that uses the Tensor Cores of the Nvidia GPU with Compute Capability 7.0 or higher.

Hardware Requirements

Nvidia GPU for Training is recommended, However, it can work with CPUs as well (Not Recommended, ImageNet is Huge. It will probably take over a year to Train)

Hardware used for Development and Testing

CPU: AMD Ryzen 7 3700X - 8 Cores 16 Threads
GPU: Nvidia GeForce RTX 2080 Ti 11 GB
RAM: 32 GB DDR4 @ 3200 MHz
Storage: 1 TB NVMe SSD
OS: Ubuntu 20.04

Minimum Hardware Requirements

CPU: AMD/Intel 4 Core CPU (CPU will become a bottleneck here)
GPU: Nvidia GeForce GTX 1660 6 GB (You can go lower, but I would not recommend it)
RAM: 16 GB
Storage: Minimum of 500 GB SSD (HDD is Not Recommended)
OS: Any Linux Distribution

Dataset Download & Setup

Enough HDD/SSD space is required for the following:

Downloading Raw Dataset - 156.8 GB
Convert to TFRecord and Store - 155.9 GB
Total Storage Required - 312.7 GB

An SSD is recommended and a Mechanical HDD should be avoided since it will slow down the data loader significantly.

Dataset Download

ImageNet Download Link: Download ImageNet Dataset

Download Train Images (Required): ILSVRC2012_img_train.tar - Size 137.7 GB
Download Val Images (Required): ILSVRC2012_img_val.tar - Size 6.3 GB
Download Train Images (Optional): ILSVRC2012_img_test.tar - Size 12.7 GB

Raw/Source Dataset Directory Structure

Download the dataset from the above link and put it in the folder like shown:

imagenet2012/
├── ILSVRC2012_img_test.tar
├── ILSVRC2012_img_train.tar
└── ILSVRC2012_img_val.tar

Processed/Destination Dataset Directory Structure

Create another folder and create the folders data, download & extracted like shown:

imagenet/
├── data/
├── downloaded/
└── extracted/

References

Source Literature: AlexNet Paper
Paper Explanation: ImageNet Classification with Deep Convolutional Neural Networks - YouTube
Download the Dataset by Signing up with your Institution Email Addreess: Download ImageNet Dataset
Tensorflow Datasets ImageNet2012 Dataset: API Docs
Efficient Data Pipelining using: Tf.Data
Tf.Data Training and Testing Split: Slice & Split Docs
Data Loader Template taken from my own Repository: Zero Coding Tf Classifier

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

Fork the Project
Create your Feature Branch (git checkout -b feature/AmazingFeature)
Commit your Changes (git commit -m 'Add some AmazingFeature')
Push to the Branch (git push origin feature/AmazingFeature)
Open a Pull Request

License

Distributed under the GNU AGPL V3 License. See LICENSE for more information.