How to Use Smart Collections to Automate Test Cycles

Smart Collections are rule-based groups of test cases that update themselves automatically. You define the criteria once, and BesTest continuously maintains the collection as your test suite changes.

The problem they solve: When your project has 20 test cases, picking the right ones for a test cycle is quick. When you have 200 or 2,000, it becomes a bottleneck. You end up asking questions like:

• •Which test cases cover the checkout module?
• •What should go into our regression suite?
• •Which high-priority tests haven't been executed this sprint?

Without Smart Collections, someone has to manually answer these questions every time a cycle is planned. That means scrolling through lists, remembering which tests belong where, and hoping nothing gets missed.

How Smart Collections work: You create a collection by defining rules (filters). BesTest evaluates those rules against your test case library and includes every test case that matches. When you add a new test case that fits the rules, it appears in the collection automatically. When you remove a tag or change a property that no longer matches, the test case drops out.

Think of a Smart Collection as a saved search that always returns current results. The collection itself does not store test cases — it stores rules. The test cases are resolved dynamically each time you view or use the collection.

This means Smart Collections require zero maintenance. You set them up once and they stay accurate as your test suite grows and changes.

Here is a concrete walkthrough for creating a regression test Smart Collection.

• •Open your Jira project
• •Click "BesTest" in the project menu bar at the top
• •Go to the Smart Collections area

• •Click "Create Smart Collection"
• •Enter a name: "Regression Suite"
• •Add a description: "All test cases tagged for regression testing"

This is where you specify which test cases belong in the collection. For a regression suite, you might set:

• •Tag equals "regression"

That single rule captures every test case in your project that has the "regression" tag. As you tag new test cases with "regression," they automatically appear in this collection.

• •Click "Save"
• •The collection view shows all test cases currently matching your rules
• •Review the list to confirm it looks right

If the list is empty, it means no test cases match the rules yet. That is expected if you have not tagged any test cases with "regression." Once you start tagging, the collection populates.

When creating a test cycle, you can now add test cases from this Smart Collection instead of selecting them individually. The collection gives you a one-click way to populate a cycle with the right tests.

A Second Example: Module-Specific Collection

Say you want a collection for your billing module:

• •Create a new Smart Collection named "Billing Module Tests"
• •Set the rule: test cases linked to requirements with component "Billing"
• •Save

Now every test case that traces back to a billing requirement is grouped automatically. When the billing team adds new requirements and test cases, the collection grows with them.

Smart Collections support multiple rule types that can be combined to create precise groupings.

Available Filter Criteria

Combining Rules

You can combine multiple rules to narrow your collection:

• •Regression + High Priority: Tag = "regression" AND Priority = "High"

Use case: A focused regression suite for time-pressured releases where you only run the most important regression tests.

• •Module + Not Executed: Component = "Payments" AND Execution Status = "Not Run"

Use case: Find test cases in the payments module that have never been executed, highlighting coverage gaps.

• •Failed Last Cycle: Execution Status = "Failed"

Use case: Quickly assemble a re-test cycle after bug fixes. Only includes tests that previously failed, saving time on re-runs.

Rule Evaluation

Rules are evaluated dynamically. Every time you open the collection or use it to populate a test cycle, BesTest re-evaluates the rules against the current state of your test cases. There is no stale data.

This is important to understand: if you remove the "regression" tag from a test case, it disappears from the Regression Suite collection immediately. No manual cleanup required.

Naming Conventions

Use clear, descriptive names that communicate both the scope and purpose:

• •Good: "Sprint Regression - Authentication Module"
• •Good: "Smoke Tests - Critical Priority"
• •Avoid: "My Collection" or "Tests 2"

Good names help the entire team understand what each collection contains without opening it.

The real payoff of Smart Collections is how they simplify test cycle creation. Instead of manually searching for and selecting test cases, you pick a collection and go.

Creating a Cycle from a Smart Collection

• •Navigate to Test Cycles in BesTest
• •Click "Create Test Cycle"
• •Fill in the cycle details (name, description, dates)
• •When adding test cases, choose to add from a Smart Collection
• •Select the collection (e.g., "Regression Suite")
• •All matching test cases are added to the cycle
• •Save the cycle

At this point, the cycle contains a snapshot of the collection's test cases at the time of creation. If new test cases match the collection rules after the cycle is created, they will appear in the collection but not in the already-created cycle. This is by design: a test cycle represents a fixed plan of what to execute.

Common Cycle Patterns

Here are patterns that teams use effectively with Smart Collections:

Full Regression Cycle

• •Collection: "Regression Suite" (all test cases tagged "regression")
• •When to use: Before every major release
• •Frequency: Once per release or sprint

Smoke Test Cycle

• •Collection: "Smoke Tests - Critical Priority" (priority = Critical)
• •When to use: After a deployment to verify core functionality
• •Frequency: After every deployment

Re-test Cycle

• •Collection: "Failed Last Cycle" (execution status = Failed)
• •When to use: After bug fixes are deployed
• •Frequency: As needed, mid-sprint

Module-Specific Cycle

• •Collection: "Billing Module Tests" (component = Billing)
• •When to use: When billing features are modified
• •Frequency: As needed based on code changes

Combining Multiple Collections

You can add test cases from more than one Smart Collection into a single cycle. For example, you might combine "Smoke Tests" and "Billing Module Tests" into one cycle when a billing change goes out. BesTest handles deduplication automatically — if a test case appears in both collections, it is only added to the cycle once.

Smart Collections are straightforward to set up, but a few practices will help you get the most value as your test suite scales.

Start with Three Core Collections

Most teams benefit from three foundational collections:

• •Regression Suite: All test cases tagged "regression." This is your safety net before releases.
• •Smoke Tests: High-priority or critical test cases. A quick check that core functionality works after a deployment.
• •Failed Tests: Test cases that failed in the most recent cycle. The starting point for re-test cycles after bug fixes.

These three cover the most common testing scenarios. Add more collections as specific needs arise.

Tag Test Cases Consistently

Smart Collections are only as good as the metadata on your test cases. If your team uses inconsistent tags (one person writes "regression," another writes "regress," a third writes "reg-test"), your collections will miss items.

Agree on a tagging convention with your team and document it. Keep tags simple:

• •regression — include in regression suite
• •smoke — include in smoke tests
• •module:billing, module:auth, module:reporting — module grouping

Review Collections Periodically

Even though collections update automatically, review them once a month:

• •Are the rules still relevant?
• •Has the team's tagging been consistent?
• •Are any collections empty or redundant?
• •Should any collections be split or combined?

A quick review prevents collection sprawl and keeps your testing workflow clean.

Do Not Over-Engineer

It is tempting to create a Smart Collection for every possible scenario. Resist this. Start with the three core collections, then add more only when a genuine need arises. Five well-maintained collections are more useful than twenty abandoned ones.

Document the Purpose

Use the description field on each Smart Collection to explain when and why it should be used. A new team member should be able to look at the list of collections and understand the testing strategy without asking anyone.