Steampunk Spotter On-prem Deployment
So you're trying out Spotter on your own air-gapped infrastructure. Wonderful!
Here's how you get started.
Quickstart
# we recommend wget as it auto-resumes interrupted downloads by default
wget -O spotter.run https://cdn.spotter.steampunk.si/public/<installer>.run
chmod u+x spotter.run
df -h # identify location of free space
./spotter.run --keep # --keep extracts into ./installer/ for potential retries
# the installer asks you for the install location (see df -h above)
# on completion, instance admin credentials are printed to stdout
firefox http://<machine-ip>/ # log in, create token
python3 -m venv .venv/ # a clean virtual environment is recommended
source .venv/bin/activate # remember to activate the venv
pip install steampunk-spotter # install the CLI
export SPOTTER_ENDPOINT=http://<machine-ip>/api # point the CLI to your on-prem instance
export SPOTTER_TOKEN=<generated-token> # auth credentials generated in the browser
spotter login # test by logging in
spotter scan <path-to-playbook>.yml # start scanning!
Note
After installation, apply the Spotter License provided to you by the Spotter team. Please refer to the License Management section for guidance.
Slowstart
With an on-premises installation of Spotter, there are a couple of extra steps as compared to using an account on Spotter SaaS. An obvious step is installing Spotter itself, but you also need to point the CLI to your installation.
Warning
Spotter on-prem is designed to be installed in a dedicated machine.
Installing Spotter on-prem on the same server as other software may have unintended side effects.
Note
Familiarity with command-line shells is necessary for installing Steampunk Spotter.
Let's see how this works.
Installing Steampunk Spotter on-prem
The team will have provided you with a link to the installer.
First, download the installer.
You can use any method you want - through the browser or in the command line.
If you're using the command line, we recommend using wget as it automatically resumes the download in case
the connection drops out.
Once the installer is transferred, mark the file as executable.
While the installer is downloading, you can check where your host has suffucient free space. Find a path in your filesystem which has sufficient free space for installation and take note, as the installer will ask you for this location later in the process.
Now run the installer.
Use the --keep argument to the installer to extract the self-extracting executable into the install/ subdirectory,
or alternatively, choose a different target with --target <dir>.
This ensures that you can re-run the installer without the lengthy decompression step in case of errors.
Tip
The installer must be run with the root user.
On some distributions, it is sufficient to run with sudo ./spotter.run, but to ensure maximum compatibility,
switch to a login shell of the root user with sudo su -.
If the installer detects that your distribution requires this step, it will raise an error with instructions.
The installer asks you a single question: where to install Spotter.
Choose any path where you have sufficient free space as identified before with df -h.
Installation takes a while, and you can monitor progress in the console. In case of errors, a verbose log file is generated and its details are printed to the console for debugging.
Upon successful installation, an instance admin is automatically created and the credentials printed into the console.
Note
Save the instance admin credentials in your password manager for later use.
Open your browser and access the Spotter host's IP address via HTTP or HTTPS and log in with your instance admin credentials.
After installation, apply the Spotter License provided to you by the Spotter team. Please refer to the License Management section for guidance.
Setting up the CLI
To get started with the CLI, a comprehensive guide is available in the CLI documentation. This section includes specifics for usage with an on-prem installation of Spotter.
First, start by installing steampunk-spotter via pip on your development machine - not on the Spotter server.
We recommend using a virtual environment to prevent conflicting dependencies between applications.
This is also the recommended way of installing packages via pip.
To use the CLI, we recommend using an API token. Generate one in Spotter in your profile page and save it. Then, set the following environment variables:
Then, verify you can log in and start scanning!
License Management
Upon your first launch, you'll need to add your Steampunk Spotter license to your on-prem deployment in the Spotter app.
Note
This applies to instance admins only.
Navigate to Admin > License. Expand the sidebar if collapsed.
Clicking the Update license button opens the Update license dialog box.
Click Upload license to open your system's File dialog, and upload your license. Alternatively, you can click the empty text box and paste the contents of the license file. Then click Save license to save your settings.
Once your license key is entered, Spotter will verify the integrity and authenticity of the license file. Successful verification unlocks the capabilities assigned to your subscription.
In case of issues with license verification, please ensure the license file has not been tampered with and is correctly placed within the designated on-premises server. For further assistance, contact support with your error logs.
With your license verified, your Spotter on-prem deployment is now ready. Proceed to administer your instance, configure additional settings, and start scanning.
LDAP Settings
This section will walk you through the process of enabling LDAP integration within the Steampunk Spotter instance, ensuring a seamless connection and mapping for users and organizations.
Steampunk Spotter encompasses a structured model where users are associated with organizations, each assigned specific roles within these entities:
- Organization admin
- Standard user
A group in LDAP is equivalent to an organization in Steampunk Spotter. In addition to this hierarchical arrangement, we have an 'admin' attribute at the instance level, designated to a select group of users, granting them elevated privileges and oversight capabilities.
Spotter uses the organization's name as a key. Users from multiple LDAP groups are merged into one organization when they are mapped to organizations that share the same name.
General LDAP Settings
- Enable LDAP Integration
To begin, you'll need to activate LDAP integration in your settings. This feature connects our system to your LDAP server, allowing for the synchronization of user and organization data. You enable it by ticking the Enable LDAP integration box.
- Sync cronjob enabled
You can enable the cronjob sync by ticking the Sync cronjob enabled. Specify the desired interval (seconds).
Connection Settings
Connection Display Name
This name will appear as a choice for the LDAP log in on the Spotter App's login page.
LDAP Host
Set the address of your LDAP server.
LDAP Port
Set your LDAP port. The default port for LDAP is 389. Ensure this port is open and accessible on your LDAP
server. The default port for secure LDAP connections using SSL/TLS is typically 636, ensuring encrypted data
transmission between client and server for enhanced security.
Authentication Mode
Set your preferred method of authentication. If your LDAP server uses TLS (Transport Layer Security), you might need additional configuration to establish a secure connection.
- Plain: transmits credentials unencrypted, necessitating a secure SSL/TLS channel to mitigate security risks inherent to this authentication method.
- Simple TLS: employs SSL/TLS protocols to encrypt communication, enhancing security by ensuring data
confidentiality and integrity between client and server. This is commonly called the
ldaps://protocol. - STARTTLS: upgrades an existing plain connection to a secure one by dynamically initiating TLS encryption, providing a flexible approach to enhancing data security during transmission.
Ignore TLS Certificate Errors
Toggle this option if you want to bypass TLS certificate validation. Use with caution, as this might expose connections to security risks.
Bind User DN
Set the Distinguished Name (DN) for binding to the LDAP server.
Example:
Bind User Password
Set the password associated with the bind user DN.
Save your settings by clicking the Save settings button. Test the connection by clicking the Test connection button.
User Mapping Settings
The User mapping tab lets site admins specify one or more blocks, each of which:
- Defines the search and filter expressions to obtain entries in LDAP, and
- Specifies what LDAP properties for users correspond to key properties in Steampunk Spotter for the mapped user.
Search DN
Set the Search DN.
Example:
Search Filter
Set the search filter to query users within the LDAP directory.
Example:
User Identifier Attribute
Steampunk Spotter uses this property as the user's user name during log-in.
Example:
User Email Attribute
Set the user email attribute that uniquely identifies the email of users in your system.
Example:
Enable the email mapping by ticking the Enable email mapping setting. If users in your LDAP don't have such an attribute, uncheck this option.First Name Attribute
Set the first name attribute that uniquely identifies the first name of users in your system.
Example:
Last Name Attribute
Set the last name attribute that uniquely identfies the last name of users in your system.
Example:
Add another mapping by clicking the Add another mapping button. Save your settings by clicking the Save settings button. Test user mapping by clicking the Test user mapping button.Organization Mapping Settings
The Organization Mapping tab lets instance admins specify one or more blocks, where each one:
- Defines the search and filter expressions to obtain organization entries in LDAP.
- Specifies how the LDAP's organization name transforms into the corresponding Steampunk Spotter organization name.
- Specifies how to obtain members of the LDAP group to become members of the Steampunk Spotter organization.
- Defines the permission level (role) that the matching users will assume in Steampunk Spotter's organization.
Normally, an LDAP group contains all users that will have the same permission level within a Steampunk Spotter organization. Therefore, the admin needs to populate two boxes that map to the same Steampunk Spotter organization name, while sourcing users from two different LDAP groups.
Only LDAP users obtained from filters in the User Mappings tab will obtain their corresponding Steampunk Spotter accounts.
Permission Level
Choose your permision level:
- Standard Member
- Organization Admin
Group Search DN
Set the base DN for searching groups within the LDAP directory.
Example:
Group Search Filter
Set the LDAP filter to query the group search results.
Example:
Group Membership Attribute
Set the LDAP attribute that represents a membership entry in an LDAP group.
Example:
Name Transformation Regex
The name transformation regex is an organizational attribute that extracts the organization name from specific data provided by LDAP.
It is a Python-style regex with a capturing group named spotter_name. This group's match is then used
as the organization name.
Example:
Add another mapping by clicking the Add another mapping button. Save your settings by clicking the Save settings button. Test group membership mapping by clicking the Test group membership mapping button.
Substitution Pattern
Declare the pattern that will result in the Spotter group name during mapping.
It is a Python-style (re.sub) replacement string. It has the \g<spotter_name> capturing group available for replacement.
Example:
Instance Admin Mapping Settings
Only users from the User Mappings tab are eligible for promotion into instance admins. If a user does not exist in the User Mappings configuration, they are not created and thus are not an instance admin.
Group Search DN
Set the base DN for searching groups within the LDAP directory.
Example:
Group Search Filter
Set the LDAP filter to query the group search results.
Group Membership Attribute
Set the attribute name that represents the group membership in the LDAP server.
Example:
Add another mapping by clicking the Add another mapping button. Save your settings by clicking the Save settings button. Test instance admin mapping by clicking the Test instance admin mapping button.Active Directory Support
To set up an LDAP integration for a self-hosted Active Directory service, follow these steps:
-
LDAP Port Adjustment: Within the LDAP Settings section of your Steampunk Spotter instance, change the LDAP Port value from the default 389 to 3268.
-
Test Connection: After adjusting the port, use the Test connection button to verify connectivity. This ensures the application can communicate with the Active Directory server through the Global Catalog over the newly specified port.
-
User and Organization Mapping: Proceed with the configuration of User and Organization Mapping settings as outlined in the documentation. Utilizing port 3268 should not affect the mapping process.
See source for additional information.
Support
Contact the LDAP administrator or the Spotter Support team for additional assistance.
TLS Settings
This section provides guidance on configuring Transport Layer Security (TLS) for your Steampunk Spotter service. Follow these instructions to enable HTTPS traffic using your organization's certificate and private key that will be used in communication between clients (the web browser, CLI) and the Steampunk Spotter instance.
If a custom TLS certificate is not enabled, a self-signed TLS certificate is generated and used for communication over HTTPS.
Enable Custom TLS Certificate
Tick the Enable Custom TLS certificate to enable the use of your TLS certificate.
Private key (Base64)
The private key associated with your certificate.
Upload the private key file by dragging it from your file manager and dropping it into this field. Alternatively, click the Browse link and select the private key file in the Open file dialog. Standard PKI files are accepted. The contents of the uploaded file will be shown in their base64-encoded form.
For example, if your private key file looks like this:
-----BEGIN PRIVATE KEY-----
MIIJQgIBADANBgkqhkiG9w0BAQEFAASCCSwwggkoAgEAAoICAQC/Oq5wr9pTPhPI
CFT6N++HrMrDE1zAwQvUoW7XbJxyh44JI1Ukfp3SSJEi4qcEq9Q5u2EDB4VWdfen
b37lGk7YRS24GkJyy4y4a8OyP4AGvDtZu6Qs06Rckj0syjaVlN/sUs8lnR9dOvWq
...
LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUpRZ0lCQURBTkJna3Foa2lH
OXcwQkFRRUZBQVNDQ1N3d2dna29BZ0VBQW9JQ0FRQy9PcTV3cjlwVFBoUEkKQ0ZU
Nk4rK0hyTXJERTF6QXdRdlVvVzdYYkp4eWg0NEpJMVVrZnAzU1NKRWk0cWNFcTlR
NXUyRURCNFZXZGZlbgpiMzdsR2s3WVJTMjRHa0p5eTR5NGE4T3lQNEFHdkR0WnU2
UXMwNlJja2owc3lqYVZsTi9zVXM4bG5SOWRPdldxCnE4Rm1WbklGTTJyK2Mva0F1
...
Certificate (Base64)
Your TLS certificate, issued by a Certificate Authority (CA). If your setup requires a certificate chain, this file contains a bundle of concatenated certificates.
Upload the public key file by dragging it from your file manager and dropping it into this field. Alternatively, click the Browse link and select the public key file in the Open file dialog. ASCII or binary formats are accepted. The contents of the uploaded file will be shown in their base64-encoded form.
For example, if your certificate file contents look like the following:
-----BEGIN CERTIFICATE-----
MIIGKTCCBBGgAwIBAgIUHlAnRPBv6dAtSAHBvjmB1lLKHrQwDQYJKoZIhvcNAQEL
BQAwgaMxCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlhMRUwEwYDVQQH
DAxTYW4gRnJhbmNpc28xITAfBgNVBAoMGEludGVybmV0IFdpZGdpdHMgUHR5IEx0
ZDEcMBoGA1UEAwwTc3BvdHRlci5leGFtcGxlLmNvbTEnMCUGCSqGSIb3DQEJARYY
aW5mb0BzcG90dGVyLmV4YW1wbGUuY29tMB4XDTI0MDIyMTExMjEzNloXDTI1MDIy
MDExMjEzNlowgaMxCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlhMRUw
Then this will appear in the Certificate (Base64) field as:
LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUdLVENDQkJHZ0F3SUJBZ0lV
SGxBblJQQnY2ZEF0U0FIQnZqbUIxbExLSHJRd0RRWUpLb1pJaHZjTkFRRUwKQlFB
d2dhTXhDekFKQmdOVkJBWVRBbFZUTVJNd0VRWURWUVFJREFwRFlXeHBabTl5Ym1s
aE1SVXdFd1lEVlFRSApEQXhUWVc0Z1JuSmhibU5wYzI4eElUQWZCZ05WQkFvTUdF
bHVkR1Z5Ym1WMElGZHBaR2RwZEhNZ1VIUjVJRXgwClpERWNNQm9HQTFVRUF3d1Rj
M0J2ZEhSbGNpNWxlR0Z0Y0d4bExtTnZiVEVuTUNVR0NTcUdTSWIzRFFFSkFSWVkK
...
Click Save & Apply Settings to save your settings. The TLS configuration is applied immediately, so no further action is required.
Support
For additional support or questions regarding TLS configuration, please contact the Spotter Support team.
Troubleshooting
Installation finished, but I missed the instance admin credentials
If you miss the information about the newly created instance admin credentials, they may be recovered from the installer
log file.
The location and format of the log files is /tmp/spotter-installer-DATETIME.log.
I have lost the log file and I don't have the instance admin credentials
It is possible to create a new instance administrator account.
Log in as root to the host of your Spotter instance.
Then change directory into your chosen installation destination (/srv by default). Then execute the following
commands:
cd docker-services/onprem/
docker compose -f docker-compose-scanner.yml exec cli ./dev.py --skip-dotenv admin instance-admin create
The command will create a new instance admin user and print the new randomly created credentials.
If you would like to supply your own credentials, include --username USERNAME --password PASSWORD to set specific
credentials.
Warning
Please note that logins into instance admins count towards the maximum users license quota.