Complete Cloud, Monitoring, Terraform, and SIEM Tutorials for Linux Admins - Textnotes

Complete Cloud, Monitoring, Terraform, and SIEM Tutorials for Linux Admins

Learn AWS SysOps, Terraform, Prometheus, Grafana, ELK Stack, and SIEM (Splunk/QRadar) with detailed explanations and real admin examples to advance your Linux/DevOps skills.

1. AWS SysOps Basics (Linux Admin-Oriented)

AWS SysOps focuses on deploying, managing, monitoring, and optimizing cloud infrastructure.

1.1 EC2 Monitoring Commands

Check CPU metrics using AWS CLI


aws cloudwatch get-metric-statistics \
--namespace AWS/EC2 \
--metric-name CPUUtilization \
--dimensions Name=InstanceId,Value=i-02acbbd8d \
--statistics Average \
--period 300 \
--start-time 2025-01-01T00:00:00Z \
--end-time 2025-01-01T01:00:00Z

1.2 Manage EBS Volumes

List Volumes


aws ec2 describe-volumes

Extend EBS Volume


aws ec2 modify-volume --volume-id vol-abc123 --size 50

Then extend filesystem:


sudo growpart /dev/xvda 1
sudo resize2fs /dev/xvda1

1.3 S3 Lifecycle Policies

Automatic archival to Glacier:

JSON Example


{
"Rules": [{
"ID": "ArchiveLogs",
"Prefix": "logs/",
"Status": "Enabled",
"Transitions": [{
"Days": 30,
"StorageClass": "GLACIER"
}]
}]
}

Apply using:


aws s3api put-bucket-lifecycle-configuration \
--bucket muni-prod-bucket \
--lifecycle-configuration file://policy.json

1.4 CloudWatch Alarms

Create CPU Alarm


aws cloudwatch put-metric-alarm \
--alarm-name HighCPU \
--metric-name CPUUtilization \
--namespace AWS/EC2 \
--statistic Average \
--period 300 \
--threshold 80 \
--comparison-operator GreaterThanThreshold \
--dimensions Name=InstanceId,Value=i-1234abcd \
--evaluation-periods 2 \
--alarm-actions arn:aws:sns:ap-south-1:1234:HighCPU

2. Terraform Basics

Terraform is Infrastructure as Code (IaC) used to create AWS resources via code.

2.1 Install Terraform


sudo yum install terraform -y

2.2 Terraform Workflow

  1. Write code (.tf files)
  2. Initialize
  3. Plan
  4. Apply
  5. Destroy

2.3 Basic Terraform Example – Launch EC2

main.tf


provider "aws" {
region = "ap-south-1"
}

resource "aws_instance" "web" {
ami = "ami-0e2ff28bfb72a4e45"
instance_type = "t2.micro"
}

Run the Terraform Commands


terraform init
terraform plan
terraform apply -auto-approve
terraform destroy -auto-approve

2.4 Variables Example

variables.tf


variable "instance_type" {
default = "t2.micro"
}

Use variable:


instance_type = var.instance_type

3. Monitoring Tools

3.1 Prometheus (Metrics Collection)

Prometheus is used to collect server and application-level metrics.

3.1 Install Prometheus (Linux)


sudo useradd --no-create-home --shell /bin/false prometheus

Download Prometheus:


tar -xvf prometheus*.tar.gz
sudo mv prometheus /etc/prometheus

3.2 prometheus.yml Example


global:
scrape_interval: 10s

scrape_configs:
- job_name: "linux"
static_configs:
- targets: ["localhost:9100"]

3.3 Install Node Exporter


node_exporter --web.listen-address=":9100"

Prometheus automatically collects:

  1. CPU
  2. RAM
  3. Disk usage
  4. Load average
  5. Network metrics

3.2 Grafana (Dashboard Visualization)

Grafana is used to visualize metrics from Prometheus, Loki, Elastic, MySQL, etc.

Install Grafana


sudo yum install grafana -y
sudo systemctl enable --now grafana-server

Access UI:


http://<server-ip>:3000

Create Dashboard Example

  1. Add Data Source → Prometheus
  2. Create Dashboard → Add New Panel
  3. Query:

node_cpu_seconds_total

You can:

  1. Monitor CPU usage
  2. Track memory usage
  3. Monitor system load
  4. Create custom alerts

3.3 ELK Stack (Elasticsearch, Logstash, Kibana)

ELK is widely used for log aggregation and analysis.

ELK Architecture

ComponentRole
ElasticsearchStores logs
LogstashIngest, transform logs
KibanaVisualize logs

Install Filebeat (Send Logs to ELK)


sudo yum install filebeat -y

Enable system logs module:


sudo filebeat modules enable system

Start service:


sudo systemctl start filebeat

Logstash Pipeline Example

/etc/logstash/conf.d/syslog.conf


input {
beats {
port => 5044
}
}
filter {
grok {
match => { "message" => "%{SYSLOGBASE}" }
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
}
}

4. SIEM Basics (Security Monitoring)

SIEM = Security Information & Event Management

Used for:

  1. Log correlation
  2. Incident detection
  3. Security monitoring
  4. Alerting

4.1 Splunk Basics

Install Splunk


rpm -i splunk.rpm
/opt/splunk/bin/splunk start --accept-license

Add Log Source


/opt/splunk/bin/splunk add monitor /var/log/messages

Search Query Example (SPL)

Search SSH login failures:


index=os_logs "Failed password"

Count per user:


index=os_logs "Failed password" | stats count by user

4.2 IBM QRadar Basics

QRadar is used for advanced threat detection.

Add a Linux Log Source

Configure:


/etc/rsyslog.conf

Add:


*.* @<qradar-ip>:514

Restart:


systemctl restart rsyslog

Basic QRadar Query (AQL)


SELECT * FROM events WHERE username = 'root'

Search failed SSH:


SELECT * FROM events WHERE QIDName='SSH Authentication Failed'