Collaborative Threat Intelligence Sharing Network

Features

• Single feed aggregator reading from known threat intelligence sources.
• Communication: Secure channels (TLS) and message queues for multi-organization data.
• Time & Consensus: Achieve consensus on new threat indicators to prevent tampering
• Non-Functional Requirements: High reliablity, robust security and scalability
• Replication and consistency: Maintain identical threat databases at all participant nodes
• Resource management & Scheduling: Balance load for analyzing huge logs from each partner
• System Architecture: Microservices for data ingestioon, verification, distribution etc.
• Middleware: Role-based access control, encryption, logging
• Big Data Processing: Analyze large scale threat patterns historically
• LLM features: generate executive summaries of the most recent threats
• Edge-Cloud Continuum: Local threat detection at teach organization's edge; aggregated intelligence in the cloud

Architecture

asd

Deployment

The repository contains source code and pipelines both for the deployments and the application itself. The kuberentes resources are managed via ArgoCD, while the software is built using github actions and pushed to dockerhub.

Repository structure

|  .github/workflows/build-and-deploy.yml
├── deployments
│   ├── apps
│   ├── threat-processor
│   ├── threat-reporting
│   └── values
├── docs
├── federated_services
│   ├── threat_processor
│   └── threat_reporting
├── node_services
│   ├── core_services
│   └── edge_services
└── tools

The repository has both the deployment declarations and the source code for the software, as well as the pipelines. The github actions are declared in (build-and-deploy.yml)[./.github/workflows/build-and-deploy.yml] and they are used to build the dockerized services and push them to dockerhub. The dockerhub repositories are public. The registries are here:

• threat_processor
• threat_reporting

The (deployments)[./deployments] -folder contains ArgoCD manifests, kubernetes yaml and helm chart value files for deploying the necessary tools. See (ArgoCD)[#argocd] for details

federated_services -folder has the source code and Dockerfile for each centralized service managed and scaled by the threat network.

node_services has the source code and Dockerfiles for each edge service, to be ran at customer premices.

tools has some debugging and testing tools.

Load balancing

HAProxy on the node LB1 is used for load balancing. It handles two endpoints: the K8S api (port 6334), and the Cilium ingress (ports 80 and 443).

K8S api proxy configuration:

frontend apiserver
    bind 0.0.0.0:6443
    mode tcp
    option tcplog
    default_backend apiserverbackend

backend apiserverbackend
    option httpchk

    http-check connect ssl
    http-check send meth GET uri /healthz
    http-check expect status 200

    mode tcp
    balance     roundrobin

    server master1 master1:6443 check verify none
    server master2 master2:6443 check verify none
    server master3 master3:6443 check verify none

Cilium shared ingress proxy configuration:

# Frontend for HTTPS (port 443)
frontend http_frontend
    bind *:80
    default_backend k8s_ingress_backend

# Frontend for HTTPS (port 443) with TLS termination
frontend https_frontend
    bind *:443 ssl crt /etc/ssl/ds.isota.io.pem
    default_backend k8s_ingress_backend

backend k8s_ingress_backend
    balance roundrobin
    option tcp-check
    # Replace these with the IPs of your Kubernetes nodes
    server master1 master1:8080 check
    server master2 master2:8080 check
    server master3 master3:8080 check

The certificate, /etc/ssl/ds.isota.io.pem, we use is a wildcard cert *.ds.isota.io from letsencrypt. It's three month duration is enough for the project so no automatic renewal was set up.

The traffic is proxied within the private network provided by cPouta. Local DNS is set up in hosts files:

## /etc/hosts
192.168.1.77	lb1 ds.isota.io
192.168.1.149	master1
192.168.1.15	master2
192.168.1.75	master3

DNS

We used a pre-existing but self-hosted and controlled domain name and DNS server setup. It runs two instances of (knot-dns)[https://www.knot-dns.cz] in master-slave mode. The name servers are:

ns2.isota.io.		3600	IN	A	94.237.113.20
ns2.isota.io.		3600	IN	AAAA	2a04:3540:1000:310:b055:ddff:fe46:d49
ns1.isota.io.		3600	IN	A	94.237.11.31
ns1.isota.io.		3600	IN	AAAA	2a04:3545:1000:720:b055:ddff:fe46:1277

The following A-records are in place for the project, and they both point to the load balancer's public IP address.

ds.isota.io.		2323	IN	A	195.148.23.15
*ds.isota.io.		2323	IN	A	195.148.23.15

Kubernetes

The centralized services are deployed in a kubernetes cluster deployed in cPouta. The configuration runs a single load balancer node with a public IP, and three nodes that act as both workers and masters due to constrained resources available. The nodes all run Ubuntu 24 LTS. The cluster was bootstrap with kubeadm. To get to a stage where ArgoCD deploys itself, a few steps were required. In short, they were:

1. Bootstrap a master node with kubeadm. Set option --skip-phases=addon/kube-proxy to let Cilium handle this functionality later
2. Set up load balancing for accessing the K8S api outside the cluster
3. Install Cilium using helm and values.yaml
4. Install ArgoCD using helm and values.yaml
5. Set up this github repository structure
6. Deploy ArgoCD from within ArgoCD and point to this repository
7. Deploy Cilium from ArgoCD and point to this repository
8. Add rest of the deployments

ArgoCD

The kubernetes deployments are under deployments -folder. The apps -folder has the ArgoCD application resources, which define what to deploy using ArgoCD. Including ArgoCD itself, here's a full list of helm charts and applications deployed:

Component	File Link	Description
ArgoCD	ArgoCD	Continuous delivery tool for managing Kubernetes deployments. https://cd.ds.isota.io/
Cilium	Cilium	Networking and security plugin for Kubernetes.
Grafana	Grafana	Dashboard and visualization platform for monitoring. https://grafana.ds.isota.io
MongoDB	MongoDB	Document-based NoSQL database for storing data.
Prometheus	Prometheus	Monitoring and alerting system for clusters.
Rook Ceph Cluster	Rook Ceph Cluster	Manages a Ceph storage cluster for distributed persistent storage.
Rook Operator	Rook Operator	Operator for orchestrating Ceph cluster operations.
Threat Processor	Threat Processor	Processes threat intelligence data for analysis.
Threat Reporting	Threat Reporting	Generates reports on detected security threats.

Metrics

Many of the services expose standard prometheus metrics to be collected. Within the cluster, they are collected with the following annotations:

spec:
  template:
    metadata:
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/path: "/metrics"
        prometheus.io/port: "9090"

For examples, see the METRICS.md