ProjectDiscovery is best known for creating open-source security tools like Nuclei, Subfinder, and HTTPx that are widely used and appreciated by the security community. Their approach to tool development aligns with the well-known "Unix Philosophy". But, what exactly does this mean?
The Unix Philosophy
The Unix Philosophy is a widely accepted set of principles in the development process that defines how development should be approached, keeping a few core principles in consideration, including:
- Simplicity: The code/program should be made to do one thing well and avoid unnecessary complexity.
- Modularity: The code/program should act as an independent component and be easily combined or re-used with other programs as required.
- Extensibility: The code/program should have scope for customisation, and features could be improved or extended with new functionalities.
If you've spent much time in a Linux terminal, you've probably experienced the Unix Philosophy first-hand! The core principles mentioned above are self-explanatory regarding their importance in the development process. By embracing simplicity, modularity, and extensibility, developers can create programs that are easier to use, maintain, and extend. These principles also encourage a collaborative and community-driven approach to development, as users can contribute their own improvements and plugins to the codebase.
ProjectDiscovery tools and the Unix Philosophy
ProjectDiscovery has developed multiple security tools that follow the Unix Philosophy's core principles, including the tools like Nuclei, Subfinder, HTTPX, Katana, etc. Let’s understand this with an example:
Let's take a closer look at the example of Subfinder and Nuclei to see how they embody the core principles of the Unix Philosophy:
- Subfinder embodies the simplicity principle by being a tool specifically designed for subdomain enumeration, which produces a simple list of subdomains that can be used as input for various other tools, including Nuclei.
- Nuclei also follows the simplicity principle as a tool designed for vulnerability scanning, and its output can be used with automated reporting, regression testing, notifications, and other workflows as needed.
- Both Subfinder and Nuclei demonstrate the modularity principle by being designed to work well with other tools, and their outputs can be integrated into various workflows.
- The extensibility principle is also evident in both tools. Subfinder is an open-source tool with ongoing development, including adding new enumeration sources and features. Nuclei allows anyone to create custom templates or modify the tool to support additional features.
It is exciting to see how ProjectDiscovery has carefully planned and targeted their tools to fit in the industry adhering various standards including the Unix Philosophy.
Simplicity principle in ProjectDiscovery Tools
The simplicity principle says “The code/program should be made to do one thing well and avoid unnecessary complexity.”
One way that ProjectDiscovery tools embrace simplicity is by focusing on specific tasks and implementing all possible solutions around that task. For example, Nuclei provides a set of pre-built templates for scanning web applications, each designed to target a specific type of vulnerability or misconfiguration. This makes it simple for users to get started with Nuclei and avoid the complexity of configuring a scanner from scratch. Similarly, Subfinder's command-line interface is straightforward, making it an accessible tool for subdomain enumeration.
The mind map showcases several widely used ProjectDiscovery tools, each of which embodies the Unix Philosophy's simplicity principle by being specifically designed to perform a single task with maximum efficiency.
Let’s understand this further with another example of “Subdomain enumeration” which is a crucial aspect of reconnaissance, and it can be performed actively or passively. Passive enumeration leverages third-party services to get the data, while active enumeration involves contacting the target's services directly, such as brute forcing DNS records. While passive enumeration involves leveraging various sources like Shodan, GitHub, and certificate lookup, most tools designed for subdomain enumeration only support specific sources or a limited number of them.
Fortunately, Subfinder breaks this mold by providing efficient and comprehensive passive subdomain enumeration capabilities. With support for a wide variety of sources that can be added with relevant API keys, Subfinder simplifies the task of subdomain enumeration, aligning with the principle of performing one task efficiently. This value is shared across other ProjectDiscovery tools, making them indispensable assets in any cybersecurity professional's toolkit.
Modularity principle in ProjectDiscovery Tools
The Modularity principle says “The code/program should act as an independent component and be combined or re-used with other programs as required.”
As we covered from the Simplicity principle that each ProjectDiscovery tools focused on a single specific tasks, let’s see how it adheres with the modularity principle using below mind map:
The beauty of ProjectDiscovery tools lies in their ability to integrate seamlessly and function as modules in an automation pipeline. This is exemplified in the mind map above, where a subdomain enumeration is performed on user-supplied input, returning a comprehensive list of subdomains. This list is then fed into HTTPX, which performs HTTP probing to identify live hosts. The resulting live hosts can be further fed into Nuclei for vulnerability scanning, with the identified vulnerabilities automatically reported or notified using the Notify tool.
This modular approach not only streamlines the automation process but also allows for flexibility and customization, with each tool acting as a building block that can be easily integrated into a larger workflow
Extensibility principle in ProjectDiscovery Tools
ProjectDiscovery tools are open-source, which allows for constant improvement and community contributions. This means that anyone can suggest new features or make improvements by submitting a feature or pull request, making the tools more efficient and versatile over time.
A perfect example of this is Nuclei, which has seen significant improvements through community contributions for the Nuclei Templates, as shown in the figure below. This demonstrates the tool's commitment to the extensibility principle, providing users with the ability to tailor the tools to their specific needs and workflows.
By encouraging community contributions, ProjectDiscovery tools ensure that they remain relevant and adaptable to changing needs, staying at the forefront of the rapidly evolving cybersecurity landscape.
Conclusion
ProjectDiscovery's approach to software development is in-line with the Unix Philosophy and has led to the creation of powerful and user-friendly security tools. By embracing simplicity, modularity, and extensibility, ProjectDiscovery tools are more efficient, flexible, and resilient, enabling security professionals, developers and organizations to perform their tasks more effectively.
The focus on one task at a time allows for deep expertise and understanding of each aspect of the security process. This approach also makes it easier for users to learn and master the tools, increasing their overall productivity. In conclusion, the Unix Philosophy remains a powerful and relevant approach to software development, and ProjectDiscovery tools serve as a prime example of its principles in action.
If you are interested to know what’s new cooking at ProjectDiscovery, make sure to sign up to the Community Newsletter - https://projectdiscovery.io/#/community