I’m sick of 1000-line Python scripts.

That cookie looks so good.

Writing scripts and tools is part of the job as a penetration tester. When we are done, we often want to share them with someone. Little one scripts are easy to throw up as GitHub gists, but that doesn’t work well when you start building complex utilities.

Our team does a lot of really weird things here at Sprocket, and we need utilities to support it with how bleeding edge™ we are at times. Even more so, we need tools that we can parse easily to support our attack surface management and continuous penetration testing solution. With that, I need a quick way to build complex CLI tools and get them shipped out to the rest of the team or share publicly.

Introducing Cookiecutter

When I was looking for a way to do this I stumbled across an old project from byt3bl33d3r a couple years back.

My Python Cookiecutter project template

Contribute to byt3bl33d3r/pythoncookie development by creating an account on GitHub.

icon-github: https://github.com/byt3bl33d3r/pythoncookie

This project is a template that utilizes the cookiecutter project.

A cross-platform command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, C projects.

A cross-platform command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, C projects

icon-github: https://github.com/cookiecutter/cookiecutter

What is cookiecutter, you might ask? In simple terms, Cookiecutter is a way to automate the setup of your project boilerplate. Think of Cookiecutter as a command-line tool that builds out all the folders, files, and basic configs needed to kick off a new project. No more copy-pasting from an old repo or hunting down what dependencies you need. You set up a template once, and then it handles the rest, getting you straight to the part that matters—writing the code that does the cool stuff.

Like I said, in the pen-testing world, you're probably cranking out scripts, tools, and random utilities on the regular. Cookiecutter can make this way smoother. Instead of setting up your file structure and dependencies for every single tool or project, you can just run a single command. Boom—ready to go with a clean, consistent setup. It’s especially great when you’re trying to maintain some level of sanity across different tools you and your team are using, so everyone’s not spending half their time figuring out someone else’s structure.

My adaptation

Over the years, I have modified byt3bl33d3r’s project pretty heavily to meet my needs. Before we jump into the details, I’m going to get out of the way and let you use it.

First, install pipx if you don’t already have it.

GitHub - pypa/pipx: Install and Run Python Applications in Isolated Environments

Install and Run Python Applications in Isolated Environments

icon-github: https://github.com/pypa/pipx?tab=readme-ov-file

Then, install cookiecutter and poetry.

pipx install cookiecutter poetry

Create a new project with cookiecutter and snickerdoodle (see what I did there?) like this:

cookiecutter gh:puzzlepeaches/snickerdoodle

You will be prompted for some variables such as tool name and information that will be needed to fill up your pyproject.toml and README.md. Boom, you should be good to go. Hop into your favorite IDE and start building out a tool by modifying __main__.py and adding files to the lib/ directory as needed.

To actually test the tool out and interact with it on the CLI, you will need to cd into your project and run the following for quick access.

poetry shell && poetry install

If you are so inclined, continue on to see how the whole thing works and the features snickerdoodle affords to users.

Cookiecutter Setup File

The cookiecutter.json is used to generate a project based on a predefined template. In the snickerdoodle project, the following variables are made available upon initial project generation.

• repo_name: The name of the repository. This is used in URLs, directory names, and anywhere the repository name is required.
• app_name: The name of the application, derived from repo_name but formatted to be suitable for Python package names (lowercase, no spaces).
• app_short_name: An optional shorter name for the application, useful for command-line interfaces or where a shorter identifier is needed.
• author, email: Information about the project's author, used in documentation and setup files.
• github_user: The GitHub username, used to construct URLs for the repository and homepage.
• version: The initial version of the application.
• copyright_year: Automatically set to the current year, used in license files and headers.
• python_version: Specifies the Python version compatibility, used in dependency files like pyproject.toml.
• logging_format: Specifies the preferred logging formats, which can be "Standard" or "JSON". This affects how logging is configured in the application.

Poetry Management

From the get-go, we have integrated Poetry for dependency management and packaging. Poetry simplifies dependency management and packaging in Python. It allows you to declare the libraries your project depends on and it will manage (install/update) them for you.

Poetry uses the pyproject.toml file to replace the traditional setup.py, requirements.txt, and MANIFEST.in in one fell swoop. This means you can manage your project dependencies, packaging, and configuration in one place, which is a massive win for simplicity and maintainability.

We have some initial libraries that are always included in every project. These are:

• validators - all in one data validation library
• click - A “Command Line Interface Creation Kit” that just works
• click_extra - Adding missing things to click
• rich - Makes your tools look pretty
• black - Makes your code look pretty

In addition, we support easy development and deployment of the package to a remote repository to GitHub and the Python package index (pypi). Take a look at the poetry documentation below for more information.

Basic usage | Documentation | Poetry - Python dependency management and packaging made easy

Basic usage For the basic usage introduction we will be installing pendulum, a datetime library. If you have not yet installed Poetry, refer to the Introduction chapter. Project setup First, let’s create our new project, let’s call it poetry-demo: poetry new poetry-demo.

Pre-configured CLI with Click

The template is pre-configured to use Click, a Python package for creating command line interfaces in a composable way with as little code as necessary.

Typer fans can kick rocks.

It’s highly configurable but comes with sensible defaults out of the box.

<code class="language-python">import click
from click_extra import config_option

from .utils.data import configure_logging, log, print
from .utils.helpers import *

# Setting context settings for click
CONTEXT_SETTINGS = dict(help_option_names=["-h", "--help", "help"])

@click.command(no_args_is_help=True, context_settings=CONTEXT_SETTINGS)
@config_option(default="config.yaml", help="Path to config file.")
@click.option("--debug", "-d", is_flag=True, help="Enable debug mode.")
def main(debug: bool) -> None:
 """{{cookiecutter.app_name}}"""
 configure_logging(debug)
 pass


if __name__ == "__main__":
 main()</code>

This setup includes basic command line options and the ability to handle configuration via a YAML file, making it easy to tweak the behavior of your CLI tool without needing to dive into the code. The click_extra package gives us instant config file support without having to build a parser our selves.

Rich Logging and Print Statements

For output and logging, we use the Rich library. Rich is a Python library for rich text and beautiful formatting in the terminal. The RichHandler for logging makes your debug output look visually appealing and more readable.

The project is configured to support both JSON and standard (rich-formatted) logging outputs. This is particularly useful for different environments; for example, JSON logging can be more suitable for output you will feed to another platform or tool.

Some key features of our rich implementation can be found below:

• The --debug flag is included in the main entry point of the application (__main__.py). This flag allows users to switch to debug logging mode right away without having to build the feature out yourself. The debug flag will subsequently update your logging config to output log.debug messages to the console.

• The function configure_logging() is called at the beginning of __main__.py to set up logging configurations based on user preferences and the environment. This function specifies whether debug logging is enabled and chooses the appropriate handler (JSON or Rich) for the context. The centralized setup ensures that all parts of the application follow the same logging standards, making logs consistent and easier to manage.

• By default, all logs are directed to a file, ensuring that you always have access to a historical record of the application's behavior. This log file is excluded from version control using the project's .gitignore file, preventing it from being pushed to GitHub.

• Throughout the project, you can call log.info, log.debug, log.error, etc., to leverage the logging capabilities, as long as you have included the necessary import statement:

  from ..utils.data import log, print
  

  This makes logging consistent and easy to use across the entire codebase.

• Besides logging, the project utilizes rich.print for pretty printing output regardless of the context. You can just print("hey casy") and the output will be pretty as long as you include the above import statement.

Extensible Utility Functions

The template includes a utils module that provides helper functions for common tasks such as validating URLs, emails, and IP addresses, reading and writing files, and more. These utilities make it easy to extend the functionality of your CLI tools without having to write boilerplate code.

Some of these are specifically in place for command line arguments you may add that are commonly included in penetration testing CLI tools. These are:

• validate_url
• validate_email
• validate_ip
• validate_domain

These can be called as click “callbacks” on your CLI arguments to make sure the user is entering the right target type for your exploit or scanning utility. To reference a callback, you can do the following in your __main__.py

@click.command(no_args_is_help=True, context_settings=CONTEXT_SETTINGS)
@config_option(default="config.yaml", help="Path to config file.")

# callback example
@click.option("--url", "-u", help="URL to validate.", callback=validate_url)
@click.option("--debug", "-d", is_flag=True, help="Enable debug mode.")
def main(debug: bool) -> None:
 """{{cookiecutter.app_name}}"""
 configure_logging(debug)
 pass

In addition, we have some functions that will read in a file, check if a file exists, return random user agents and more. Take a look for current options. Feel free to add your own, and import them throughout the app as needed.

Clean README.md Template

Have you ever envied those beautiful README.md files all over GitHub? I don’t really like them, not going to lie, so I went simple with it. Our README.md has the following structure out of the box with some predefined install instructions that are awfully partial to GitHub users 😁.

• Installation: Guides users through the process of installing the application, covering various methods like PyPi, direct Git installation, or local setup for development.
• Getting Started: Intended to provide initial guidance to help new users quickly start using the application.
• Usage: Details the commands and options available within the application, typically including examples to demonstrate common use cases. Usually I just put the help menu output here!
• Coming Soon: All the things you plan to add, but never will.
• Thanks: Someone else did it first, you know it. Credit them here!

Some variables will be automatically be filled it based on your cookiecutter variables such as the remote install location and app name.

Easy HTTP Requests

A file req.py can be called from anywhere to use sane defaults for making HTTP requests throughout your application. When imported, it supressess SSL warnings, supports backoff and retry strats, as well as allows for an optional HTTP proxy for the times where you want to send your traffic through BurpSuite for debugging. I got this from stack exchange a long time ago.

<code class="language-python">from urllib.parse import urlparse

import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry

from .setup import log, print

# Dealing with SSL Warnings
try:
 import requests.packages.urllib3
 requests.packages.urllib3.disable_warnings()
except Exception:
 pass


def requests_retry_session(
 retries=3,
 backoff_factor=0.3,
 status_forcelist=(500, 502, 504, 503, 403),
 session=None,
 proxy=None,
):
 session = session or requests.Session()
 retry = Retry(
 total=retries,
 read=retries,
 connect=retries,
 backoff_factor=backoff_factor,
 status_forcelist=status_forcelist,
 )
 adapter = HTTPAdapter(max_retries=retry)
 session.mount("http://", adapter)
 session.mount("https://", adapter)
 
 if proxy:
 session.proxies = {"http": proxy, "https": proxy}
 
 return session</code>

It’s pretty simple but nice to have, especially when you are doing things at scale or debugging some exploit code.

Wrapping Up

There is a lot more in here than I even touched on to be honest. There are pre-commit configs, vscode sane defaults and even maybe some easter eggs. The article was getting way too long, however. Dig in on the code though, you will find some fun stuff to make your life easier, I’m sure of it.

All in all though, snickerdoodle is meant to make your life easy. If you have an idea, put it into action instead of building out a codebase. I use this all the time for little Python tools all the way up to building Flask apps and frontends for presentations.

So, yeah, just download Snickerdoodle and get on with it. I mean, why waste time setting up a project from scratch when you could be doing literal hacking. We have the best job in the world, don’t forget it.