Deep Packet Inspection (DPI)

[hsing]

Hello community,

I previously asked about Deep Packet Inspection (DPI) on the Intrusion Detection and Prevention forum (https://forum.opnsense.org/index.php?topic=39944.msg195936#msg195936). I've learned that Zenarmor may offer similar functionality (though I haven't tested it yet). However, I found out that Zenarmor is not open source. Therefore, I'm inquiring here if there are any other tools that can also achieve DPI, or any directions I should explore?

I found a concept in a document on page 35 "BPF + IPFW + TAG = L7 Filter RTFM ng_tag(4)" (https://www.netbsd.org/gallery/presentations/ast/2012_AsiaBSDCon/Tutorial_NETGRAPH.pdf). Are there any individuals with similar experience or interest who would like to discuss this?

[Patrick M. Hausen]

What exactly is it you like to discuss?

My stance is that DPI is

1. mostly marketing
2. has little technical value
3. has become mostly useless with the proliferation of TLS for everything

Possibly that is why you get so few answers. Are you intending to implement DPI functionality? Are you expecting the OPNsense project to do so? Are you shopping for some organisation and DPI is one of the mandatory checkmarks?

I stand by "DPI has always been snake oil" and have little incentive to discuss it.

[hsing]

Thank you for your response. I will address your questions one by one:

    Are you intending to implement DPI functionality?
    Yes, I have been gathering relevant information for learning this week. I found an article about DPI from Zenarmor mentioning that nDPI and Netifyd are developed based on OpenDPI. Additionally, I discovered attempts at similar integrations in the past on the OPNsense forums, although it was unclear why these were discontinued (I now have some understanding from your reply).

    Are you expecting the OPNsense project to do so?
    Not exactly; my main goal was to gain insights and reference materials from the OPNsense project.

    Are you searching for a solution for some organization, and is DPI one of the mandatory requirements?
    Yes, it is a development project. Being a beginner with limited development experience, I mainly gather information through studying open-source projects. The purpose of DPI is to integrate IDS/IPS with DPI to manage the firewall's data flow, allowing packets with specific "tags" to pass while blocking malicious attacks to protect critical infrastructure. In DDoS attacks, it's essential to effectively control the influx of massive amounts of packets, ensuring normal packets are not blocked, while also reducing the firewall's load.

[Monviech (Cedrik)]

This project here already does it very efficiently:

https://www.crowdsec.net/

Its included in OPNsense too and the far better choice than DPI.

https://github.com/crowdsecurity/crowdsec

EDIT:
Also DDOS will always be a problem. For a decision to be made, the packet will always have to be received. The only efficient DDOS protection is "something" in front of the main firewall that has way more processing power to make these decisions for you and give you a filtered link. Most people use CDNs for that, like Cloudflare.

[hsing]

Thank you for your response; I will delve deeper into studying and researching this project.

Additionally, I am quite curious: since both IDS/IPS and DPI are used to inspect packets entering and exiting, with IDS/IPS checking traffic behavior and DPI inspecting the content of packets, why not combine the two to both inspect packets and analyze traffic behavior? Therefore, I have started learning how the components used in IDS/IPS handle the ingress and egress of packets and the inspection processes, and exploring how DPI processes and inspects packets to see if there is a possibility to integrate it with IDS/IPS. Could this lead to more detailed attack analysis? However, due to my insufficient knowledge base, I am currently unable to achieve this.

[Patrick M. Hausen]

In my opinion IDS/IPS is DPI. But with TLS encrypted connections there isn't much to inspect, so the technology is at somewhat a dead end. Breaking up and man-in-the-middle-ing TLS actually weakens security, so I strongly advise against doing so.

We must come to terms with the fact that the internet relies on the end-to-end principle and the traffic e.g. from your browser to your banking website must not be interfered with.

There are other measures to protect from luring users into downloading malicious content, some on the end system, some on the firewall.

[hsing]

I generally understand what you are describing.

Let me illustrate with another application scenario, suppose today we need to protect the connection management equipment from external to internal access at the bank. For this purpose, I have deployed a firewall at the front end of the device to manage the equipment and block suspicious external connections. By configuring IDS/IPS with packet analysis capabilities, we can allow packets with specific "tagged IDs" or "specific formats" to connect, while rejecting others. In this case, can DPI achieve our goal of using IDS/IPS to block packets that do not meet the specific tags or formats, while DPI checks if the packets comply with the required standards?

[bimbar]

I generally understand what you are describing.

Let me illustrate with another application scenario, suppose today we need to protect the connection management equipment from external to internal access at the bank. For this purpose, I have deployed a firewall at the front end of the device to manage the equipment and block suspicious external connections. By configuring IDS/IPS with packet analysis capabilities, we can allow packets with specific "tagged IDs" or "specific formats" to connect, while rejecting others. In this case, can DPI achieve our goal of using IDS/IPS to block packets that do not meet the specific tags or formats, while DPI checks if the packets comply with the required standards?

For that, you would have to research the topics "reverse proxy" and "loadbalancer". IDS / IPS with DPI usually is not the right tool for that job.

[Patrick M. Hausen]

@hsing you probably want a web application firewall (WAF). I am in no way debating the usefulness of those.

Besides many open source and commercial products that can be implemented on suitable systems there is one built-in to OPNsense:

https://docs.opnsense.org/manual/how-tos/nginx_waf.html

[hsing]

@bimbar @Patrick M. Hausen @Monviech Thank you for your insights. My current research focus is primarily on how OT firewalls filter packets, and thus areas like CDN, TLS encryption, and reverse proxies are currently not within my scope of study. As a result, I may need to acquire more knowledge before entering these fields. Moving forward, I will concentrate on researching the components and packages used in DPI and IDS/IPS, examining whether there is an intersection or if they need to be applied separately to achieve the goal, which is to use DPI to verify if packet contents meet specific requirements and then utilize IDS/IPS to filter incoming and outgoing packets.

Thank you again for your assistance. I will delve further into understanding the application of DPI and IDS/IPS at the system component level. I am very grateful for the information and suggestions you have provided!