Akatama's Slice of the Internet

Why on Earth Would You Run Pytest as Root?

By Jimmy Lindsey

When a project like Internet-in-a-Box calls for unconventional solutions, you have to get creative. What started as the simple addition of allowing GUI tests to be run on Firefox in addition to Google Chrome, turned into a real-world debugging adventure involving quirks with browsers, Pytest and Internet-in-a-Box's own setup of a Python virtual environment. Along the way, we'll explore how the integration tests are constructed and figure out how to run these tests as root. If you've ever wanted a behind-the-scenes look at pragmatic DevOps problem-solving, you're in the right place!

What is Internet-in-a-Box (IIAB)?

IIAB is a project that is built to bring offline knowledge to remote communities. There are a lot of small villages out there that do not have a great internet connection, and so having a server nearby that people can remotely connect to with their phones means that these communities can still enrich themselves with knowledge. You can also imagine how a parent would want control over what type of content their young children have access to.

IIAB is usually ran on a Raspberry PI, but it can be easily run on any Debian-based system¹. IIAB itself is primarily an Ansible configuration which installs and configures various software, from Tailscale to OpenStreetMap to downloads of Wikipedia and other articles. One of these projects is IIAB's fork of Calibre-Web. While the original allows people to download and organize ebooks, it does not have the same capability for videos². A lot of people who use IIAB wanted this, so it was forked.

The Original Integration Tests

The first integration tests were added to iiab/calibre-web a couple months before I joined by Hermes Ruiz, known as thotmx on GitHub. Since I am currently a SDET, I was asked to take a look and see if I could improve it. Let me talk a bit about what I found.

Test Plans

First, tests are organized into tests plans under the features directory. Currently, the only existing test plan is basic_behavior.feature.

Feature: Basic behavior 
    Testing basic behavior like showing home page and login 

    Scenario: Home Page 
        Given Calibre-Web is running 
        When I go to the home page 

        Then I should not see the error message
        And see homepage information

    Scenario: Login 
        Given I visit the Calibre-Web homepage 
        When I login with valid credentials

        Then I should see the success message
        And see the information for logged users

Each test starts with a scenario and a test name. This feature file is actually important to the written tests, as thotmx decided to use pytest-bdd. In other words, to add new tests, you would need to at least add a new scenario to this file. Depending on the test, you may need to create a new feature file.

I do not actually practice behavior driven-development much at my job. The closest would be when I convert manual test cases to automated tests, or when when I write specific end-to-end tests for APIs. I have never used a framework that ties these scenarios this deeply to the actual tests. I am still getting used to how this works, and to be totally honest, I am not sure I am a fan of it.

These two tests are currently the only ones we have. They are very simple, but very important. In the future, I hope to add more tests.

Tests

As already mentioned, these tests are implemented in pytest-bdd. However, since Calibre-Web runs on localhost:8083 and is accessed via a browser, we need a bit more to allow us to actually test this application. The plugin we use for handling the browser is pytest-splinter, which uses Selenium under the hood.

@pytest.fixture(scope='session')
def splinter_headless():
    """Override splinter headless option."""
    if os.environ['HEADLESS'] == "true":
        return True
    else:
        return False 

@pytest.fixture(scope='session')
def splinter_webdriver():
    """Override splinter webdriver name."""
    return 'chrome'

These two functions are setting up the browser to run the tests. First we check if an environment variable, HEADLESS, is true and if it is then we run Google Chrome in headless mode³. Note that at this moment in time, we could only run tests on Chrome.

@scenario('basic_behavior.feature', 'Home Page')
def test_home_page():
    """Home Page."""


@given('Calibre-Web is running')
def _(step_context):
    """Calibre-Web is running."""
    step_context['ip_address'] = 'localhost:8083'


@when('I go to the home page')
def _(browser, step_context):
    """I go to the home page."""
    url = urljoin("".join(['http://', str(step_context['ip_address'])]), '/')
    browser.visit(url)


@then('I should not see the error message')
def _(browser, step_context):
    """I should not see the error message."""

@then('see homepage information')
def _(browser):
    """see homepage information."""
    print("!!!!!!!")
    print(browser.title)
    print(browser.url)
    print("!!!!!!!")
    assert browser.is_text_present('Books'), 'Book test'

This is an example of a test named Home Page. You can see Pytest fixtures named scenario, given, when, and then which relate the test plan defined in the feature file.

To run tests, you would run the following command:

HEADLESS=true pytest -s

Adding Firefox

Firefox is the default browser in Debian and Ubuntu, so we should also make sure our tests run on that browser as well. I also considered adding Chromium, which is the default browser for Raspberry Pi OS, but Splinter (a dependency of pytest-splinter) does not support Chromium as of the time of writing. The goal is to test with the same browsers as our users, and we'll take Chrome as it is based on Chromium.

The first problem you will face if you try to use Pytest with Firefox on Ubuntu, is that Pytest simply doesn't know what to do if Firefox is installed with Snap. I tried multiple times to fix it, and the only way was to uninstall the Snap version of Firefox, and then install Firefox with the .deb package. I followed this guide for doing that, which I made sure to do after sudo snap remove firefox.

After some research on pytest-splinter, this is what I came up with:

@pytest.fixture(scope="session")
def splinter_driver_kwargs(splinter_webdriver):
    """Override Chrome WebDriver options"""
    if splinter_webdriver == "chrome":
        chrome_options = webdriver.ChromeOptions()

        # List of Chromium Command Line Switches
        # https://peter.sh/experiments/chromium-command-line-switches/
        chrome_options.add_argument("--no-sandbox")

        return {"options": chrome_options}

    elif splinter_webdriver == "firefox":
        firefox_options = webdriver.FirefoxOptions()
        return {"options": firefox_options}
    else:
        raise ValueError(
            "Invalid browser passed to --splinter-webdriver. Only Chrome and Firefox are allowed"
        )

First, I removed the splinter_headless function as pytest-splinter already provides a command-line parameter (--splinter-headless) to make the browser run in headless mode. Another command-line parameter pytest-splinter provides is --splinter-webdriver, which we can pass an argument to. As you see here, we are running Chrome in --no-sandbox mode, which will be significant in the next section. The end result is that we now run this command if we want to run the browser in regular mode:

pytest -s --splinter-webdriver firefox

And we run this if we want to run the browser in headless mode:

pytest -s --splinter-webdriver chrome --splinter-headless

It is more to type, but we have more control over it. In the end, these tests will mostly be run either with a copy and paste or in a GitHub Actions workflow, so the extra typing doesn't even really matter.

Pytest as root

The work for adding Firefox didn't take me that long, but what was a stickler was that we could not run the tests as root. For people with experience with Pytest, this is not strange at all, as it is heavily discouraged. However, when I mentioned this to the maintainer of IIAB, Adam, he was not familiar with this and was surprised.

In the end, he told me that there are a lot of non-technical people who help manually test IIAB to make sure it works. To simplify things, they have told them to create virtual machines with Multipass and to run everything as root (specifically, sudo -i). Those who are manually testing may want to run these tests, and so he wanted me to make it work.

It did take some time, but the fix for this ended up being pretty simple. It was further complicated by the fact that when IIAB installs Calibre-Web, it does so in a nonstandard way. Instead of cloning the repo and then creating the virtual environment directory with venv, the virtual environment directory is created, and then the repo is cloned inside. I hope that someday in the future I can change that, as it may help future contributors troubleshoot issues. In fact, this quirk is my best guess for why Firefox was having trouble running as root, because once I updated our pytest.ini file to specify the path to the tests, Firefox began to work when running as root.

[pytest]
bdd_features_base_dir = features/
testpaths = tests/functional
filterwarnings = ignore::DeprecationWarning

Chrome was a different beast. Google purposefully makes it so Chrome will not run as root, and instead will give you an unhelpful error message about --user-data-dir. However, if you look into the Chrome options, they will mention the experimental --no-sandbox. This works and allows Chrome to be run as the root user, which is why we pass it to Chrome above in our test setup.

Conclusion

I am quite happy with how everything worked out in the end. While running Pytest as root isn't ideal, sometimes meeting people where they are means bending a few best practices, especially because it made these tests more accessible for the less technical IIAB contributors. If this kind of work sounds interesting, stay tuned, as I have more IIAB adventures to share soon!