📊 Automated ELK Stack Deployment Project

The files in this repository were used to configure the network depicted below:

These files have been tested and used to generate a live ELK deployment on Azure. They can be used to either recreate the entire deployment pictured above, or select portions of the Project 1 Red-Team Network Diagram file to install specific components, such as Filebeat:

• filebeat-playbook.yml
• filebeat-configuration.yml

📋 Document Overview

This document contains the following details:

• Description of the Topology
• Access Policies
• ELK Configuration
  • Beats in Use
  • Machines Being Monitored
• How to Use the Ansible Build

🖥️ Description of the Topology

The main purpose of this network is to expose a load-balanced and monitored instance of DVWA, the Dmn Vulnerable Web Application.*

• Load Balancing: Ensures high functionality of the application while restricting high traffic.

• Security Aspects of Load Balancers:

  • Prevents server overload and optimizes productivity.
  • Adds resiliency by rerouting live traffic to eliminate single points of failure from attacks such as DDoS.

• Advantages of a Jump Box:

  • Jump boxes are highly secured computers, dedicated solely to admin tasks.
  • They minimize the chances of malware infection by maintaining a lockdown security environment.

Integrating an ELK server allows users to easily monitor vulnerable VMs for changes in network and system logs:

• Filebeat: Monitors specified log files and forwards them to Elasticsearch/Logstash for indexing.
• Metricbeat: Records metrics/statistics data and transports them to the specified output via Elasticsearch/Logstash.

🖥️ Configuration Details

The configuration details of each machine are listed below. Use the Markdown Table Generator to add/remove values from the table.

🔑 Access Policies

The machines on the internal network are not exposed to the public Internet:

• Only the Jump-Box-Provisioner can accept connections from the Internet.
• Access is only allowed from the following IP addresses:
  • 71.59.34.72 (LocalHost IP address)

Internal machines can only be accessed via Jump-Box-Provisioner:

• Allowed Access to ELK VM:
  • Machine: Jump-Box-Provisioner
  • IP Address: 10.0.0.5 (Private)

Summary of the access policies:

Note: All these VMs can only be accessed from the Jump-Box-Provisioner.

⚙️ ELK Configuration

Ansible was used to automate the configuration of the ELK machine. No manual configuration was performed, which offers the following advantages:

• Automation Benefits:
  • YAML Playbooks streamline configuration management and automation.
  • Capable of automating complex multi-tier IT application environments.

The playbook implements the following tasks:

• First, SSH into the Jump-Box-Provisioner (ssh sysadmin@23.96.123.102).
• Start/Attach to the Ansible Docker (sudo docker start -i sharp_gauss)/(sudo docker attach sharp_gauss).
• Navigate to /etc/ansible/roles directory and create the ELK playbook (Elk_Playbook.yml).
• Execute the playbook (ansible-playbook Elk_Playbook.yml).
• SSH into the ELK-VM to verify the server status.

The following screenshot displays the result of running docker ps after successfully configuring the ELK instance:

🖥️ Target Machines & Beats

This ELK server is configured to monitor the following machines:

• IP Addresses of Monitored Machines:
  • WEB-1: 10.0.0.7
  • WEB-2: 10.0.0.6
  • WEB-3: 10.0.0.8

Installed Beats on these machines:

Beats collect the following information:

• Filebeat: Collects log files from specified locations on remote machines (e.g., logs generated by Apache or MySQL).
• Metricbeat: Collects machine metrics such as CPU usage and uptime.

🛠️ Using the Playbook

To use the playbook, you need a pre-configured Ansible control node. SSH into the control node and follow these steps:

—- Filebeat —-

• Copy the filebeat-configuration.yml file to /etc/ansible/roles/files.
• Update the filebeat-configuration.yml file to include the ELK private IP in lines 1106 and 1806.
• Run the playbook and navigate to http://13.66.196.172:5601/ (ELK-VM public IP) to verify the installation.

—- Metricbeat —-

• Copy the metricbeat-configuration.yml file to /etc/ansible/roles/files.
• Update the metricbeat-configuration.yml file to include the ELK private IP in lines 62 and 96.
• Run the playbook and navigate to http://13.66.196.172:5601/ (ELK-VM public IP) to verify the installation.

🔍 Questions & Answers

• Which file is the playbook? filebeat-playbook.yml
• Where do you copy it? /etc/ansible/roles
• Which file do you update to run the playbook on a specific machine? /etc/ansible/hosts
• How do I specify the installation machine for ELK vs. Filebeat? By defining separate groups in /etc/ansible/hosts: one for webservers (Filebeat) and another for elkservers (ELK).
• What URL do I use to check if the ELK server is running? http://13.66.196.172:5601/

🎉 Bonus: Commands

———- Filebeat ————-

• To create the configuration file:

  nano filebeat-configuration.yml

• To create the playbook:
  ```bash
  nano filebeat-playbook.yml

• name: installing and launching filebeat
  hosts: webservers
  become: true
  tasks:

  • name: download filebeat deb
    command: curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.7.1-amd64.deb

  • name: install filebeat deb
    command: dpkg -i filebeat-7.7.1-amd64.deb

  • name: drop in filebeat.yml
    copy:
    src: ./files/filebeat-configuration.yml
    dest: /etc/filebeat/filebeat.yml

  • name: enable and configure system module
    command: filebeat modules enable system

  • name: setup filebeat
    command: filebeat setup

  • name: start filebeat service
    command: service filebeat start
    ```

• To run the playbook:

  ansible-playbook filebeat-playbook.yml

  ———- Metricbeat ————-
• To create the configuration file:

  nano metricbeat-configuration.yml

• To create the playbook:
  ```bash
  nano metricbeat-playbook.yml

• name: installing and launching metricbeat
  hosts: webservers
  become: true
  tasks:

  • name: download metricbeat deb
    command: curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-7.7.1-amd64.deb

  • name: install metricbeat deb
    command: sudo dpkg -i metricbeat-7.7.1-amd64.deb

  • name: drop in metricbeat.yml
    copy:
    src: /etc/ansible/roles/files/metricbeat-configuration.yml
    dest: /etc/metricbeat/metricbeat.yml

  • name: enable and configure system module
    command: metricbeat modules enable system

  • name: setup metricbeat
    command: metricbeat setup

  • name: start metricbeat service
    command: service metricbeat start
    ```

• To run the playbook:

  ansible-playbook metricbeat-playbook.yml