edgex-snap-testing

EdgeX Snap Tests

Test scripts, Github actions, and workflows for the EdgeX Foundry snaps.

The following diagram shows the Snap Testing workflow for building and testing snaps from upstream source code:

---
title: Snap Testing Workflow
---
flowchart LR
  subgraph build [Build Job]
    builda[[Build Action]] --> Source
    --> Build[Build Snap]
    --> Snap[/Artifact<br>Snap/]
  end
  subgraph test [Test Job]
    Snap -.-> testa[[Test Action]]
    --> gotests[Go Test Suites]
    --> Logs[/Artifact<br>Logs/]
  end

The Github Workflow configurations (triggers, jobs, etc) are maintained in respective upstream source codes.
The Github Actions and testing suites are maintained in this repository.

For example, this is the workflow of the tests that run on the edgex-go project. The Github Actions used in the workflow are versioned using a major semantic versioning tag. This tag is automatically moved to the latest minor and patch releases of the tests (this repository).

This project has additional workflows such as for running the tests weekly and on local PRs.

Test locally

This section includes example command to run tests.

Useful go test flags are:

• -v is to enable verbose output
• -failfast makes the test stop after first failure
• -timeout 60m extends the timeout to longer than the default 10m
• -count 1 is to avoid Go test caching for example when testing a rebuilt snap

Run one testing suite

go test -v -failfast -count 1 ./test/suites/device-mqtt

Run all suites

go test -p 1 -timeout 60m -failfast -count 1 ./test/suites/...

Run one suite with env variables

The environment variables are defined in test/utils/env.go

Full config test:

FULL_CONFIG_TEST=true go test -v -failfast -count 1 ./test/suites/device-mqtt

Testing with a local platform snap:

LOCAL_PLATFORM_SNAP="edgexfoundry_3.1.0-dev.3_amd64.snap" \
go test -v -failfast -count 1 ./test/suites/edgexfoundry

Testing with a local service snap:

LOCAL_SERVICE_SNAP="edgex-device-mqtt_2.0.1-dev.15_amd64.snap" \
go test -v -failfast -count 1 ./test/suites/device-mqtt

Testing with local platform and service snaps:

LOCAL_PLATFORM_SNAP="edgexfoundry_3.1.0-dev.3_amd64.snap" \
LOCAL_SERVICE_SNAP="edgex-device-mqtt_2.0.1-dev.15_amd64.snap" \
go test -v -failfast -count 1 ./test/suites/device-mqtt

Test with skipping the removal of snaps during teardown:

SKIP_TEARDOWN_REMOVAL=true go test -v -failfast -count 1 ./test/suites/

Test by revision:

PLATFORM_CHANNEL=4259 go test -v -failfast -count 1 ./test/suites/edgex-no-sec

This requires developer access; see snap install -h for details.

Run only one test from a suite

go test -v ./test/suites/edgexfoundry --run=TestCommon

go test -v ./test/suites/edgex-config-provider -run=TestConfigProvider/device-virtual

Test the testing utils

go test ./test/utils -count=10

Run EdgeX Ubuntu Core tests

Refer to edgex-ubuntu-core-testing

Test using Github Actions

This project includes two Github Actions that can be used in workflows to test snaps:

• build: Checkout code, build the snap, and upload snap as build artifact
• test: Download the snap from build artifacts (optional) and run smoke tests

A workflow that uses both the actions from v2 branch may look as follows:

.github/workflows/snap.yml

name: Snap Testing

on:
  pull_request:
    branches: [ main ]
  # allow manual trigger
  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Build and upload snap
        id: build
        uses: canonical/edgex-snap-testing/build@v2
    outputs:
      snap: $

  test:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - name: Download and test snap
        uses: canonical/edgex-snap-testing/test@v2
        with:
          name: device-mqtt
          snap: $

Testing Scripts

The testing scripts can be located in the ./test/scripts directory.

To create a token for example user:

./test/scripts/login-test-user.sh

To create a self-signed TLS certificate and replace the defaults:

./test/scripts/create-tls-certificates.sh