Issue in crating custom ruleset

[mahamuneeprasad]

Have you tried to resolve this issue yourself first?

• I confirm I have gone through the above steps and still have an issue to report.

Bug Description

Hello All,

Our Development team tried to crate one custom ruleset to identify if any test method does not have Test.starTest() and Test.stopTest() declaration but it did not worked. We are not getting any error.

Please note that our team referred below GitHub link as well.
https://github.com/starch-uk/sca-extra

Please see below for the TestMethodsMustUseStartStopTest.xml file as well as code-analyzer1.yaml file. Please review and help us in resolving issue. We need to resolve this issue on priority.

code-analyzer1.yaml
engines:
pmd:
disable_engine: false
file_extensions:
apex:
- .apex
- .cls
- .trigger
custom_rulesets:
- rulesets/bestpractices/FinalVariablesMustBeFinal.xml
- rulesets/bestpractices/RegexPatternsMustBeStaticFinal.xml
- rulesets/bestpractices/StaticMethodsMustBeStatic.xml
- rulesets/bestpractices/StaticVariablesMustBeFinalAndScreamingSnakeCase.xml
- rulesets/bestpractices/TestClassIsParallel.xml
- rulesets/codestyle/AvoidMagicNumbers.xml
- rulesets/codestyle/AvoidOneLinerMethods.xml
- rulesets/codestyle/InnerClassesMustBeOneWord.xml
- rulesets/codestyle/MapShouldBeInitializedWithValues.xml
- rulesets/codestyle/MultipleStringContainsCalls.xml
- rulesets/codestyle/NoMethodCallsAsArguments.xml
- rulesets/codestyle/NoMethodCallsInConditionals.xml
- rulesets/codestyle/NoSingleLetterVariableNames.xml
- rulesets/codestyle/PreferBuilderPatternChaining.xml
- rulesets/codestyle/PreferConcatenationOverStringJoinWithEmpty.xml
- rulesets/codestyle/PreferMethodCallsInLoopConditions.xml
- rulesets/codestyle/PreferNullCoalescingOverTernary.xml
- rulesets/codestyle/PreferSafeNavigationOperator.xml
- rulesets/codestyle/PreferStringJoinOverConcatenation.xml
- rulesets/codestyle/PreferStringJoinOverMultipleNewlines.xml
- rulesets/codestyle/PreferStringJoinWithSeparatorOverEmpty.xml
- rulesets/codestyle/SingleArgumentMustBeSingleLine.xml
- rulesets/codestyle/VariablesMustNotShareNamesWithClasses.xml
- rulesets/design/AvoidLowValueWrapperMethods.xml
- rulesets/design/AvoidTrivialPropertyGetters.xml
- rulesets/design/ClassesMustHaveMethods.xml
- rulesets/design/CombineNestedIfStatements.xml
- rulesets/design/EnumMinimumValues.xml
- rulesets/design/InnerClassesCannotBeStatic.xml
- rulesets/design/InnerClassesCannotHaveStaticMembers.xml
- rulesets/design/NoCustomParameterObjects.xml
- rulesets/design/NoInterfacesEndingWithCallback.xml
- rulesets/design/NoParameterClasses.xml
- rulesets/design/NoThisOutsideConstructors.xml
- rulesets/design/NoUnnecessaryAttributeVariables.xml
- rulesets/design/NoUnnecessaryReturnVariables.xml
- rulesets/design/UnusedVariableNotReturnedOrPassed.xml
- rulesets/design/PreferPropertySyntaxOverGetterMethods.xml
- rulesets/design/PreferSwitchOverIfElseChains.xml
- rulesets/design/SingleParameterMustBeSingleLine.xml
- rulesets/design/DmlInLoopAcrossMethods.xml
- rulesets/design/TestMethodsMustUseStartStopTest.xml
- rulesets/documentation/ExceptionDocumentationRequired.xml
- rulesets/documentation/MethodsRequireExampleTag.xml
- rulesets/documentation/ProhibitAuthorSinceVersionTags.xml
- rulesets/documentation/SingleLineDocumentationFormat.xml
- rulesets/documentation/ValidGroupTagValues.xml
regex:
custom_rules:
NoConsecutiveBlankLines:
regex: /\n\s*\n\s*\n/g
file_extensions: ['.apex', '.cls', '.trigger']
description:
'Prevents two or more consecutive blank lines in Apex code.
Code should have at most one blank line between statements,
methods, or other code elements.'
violation_message:
'Two or more consecutive blank lines are not allowed. Use at
most one blank line between statements.'
severity: 'Moderate'
tags: ['CodeStyle', 'Recommended']
ProhibitSuppressWarnings:
regex: /@SuppressWarnings([^)])|//\sNOPMD/gi
file_extensions: ['.apex', '.cls', '.trigger']
description:
'Prohibits the use of @SuppressWarnings annotations and
NOPMD comments in Apex code. Suppressions hide code quality
issues; prefer fixing the underlying problems or improving
rules instead.'
violation_message:
'Suppression of warnings is not allowed. Fix the underlying
issue or improve the rule instead of suppressing violations.'
severity: 'High'
tags: ['CodeStyle', 'Recommended']
NoLongLines:
regex: /.{81,}/gm
file_extensions: ['.apex', '.cls', '.trigger']
description:
'Enforces a maximum line length of 80 characters. Lines
longer than 80 characters reduce readability and make code
harder to review. Use shorter class, attribute, method, and
variable names to improve readability.'
violation_message:
'Line exceeds 80 characters. Use shorter class, attribute,
method, and variable names to improve readability.'
severity: 'Moderate'
tags: ['CodeStyle', 'Recommended']
ProhibitPrettierIgnore:
regex: ///\s*prettier-ignore/gi
file_extensions: ['.apex', '.cls', '.trigger']
description:
'Prohibits the use of prettier-ignore comments in Apex code.
Code should be formatted consistently without exceptions.
Using prettier-ignore comments undermines code formatting
standards and makes the codebase less maintainable.'
violation_message:
'Prettier-ignore comments are not allowed. Code should be
formatted consistently without exceptions.'
severity: 'Moderate'
tags: ['CodeStyle', 'Recommended']

TestMethodsMustUseStartStopTest.xml

<ruleset
name="Apex Design Rules"
xmlns="http://pmd.sourceforge.net/ruleset/2.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 https://pmd.sourceforge.io/ruleset_2_0_0.xsd"

<description>
    Custom Salesforce PMD rules to enforce usage of Test.startTest() and Test.stopTest() in all test methods
</description>

<!-- Rule: Missing Test.startTest() or Test.stopTest() -->
<rule name="TestMethodsMustHaveExactlyOneStartStopTest"
  language="apex"
  message="Test methods must contain exactly one Test.startTest() and one Test.stopTest()."
  class="net.sourceforge.pmd.lang.rule.xpath.XPathRule">

<description>
    Ensures every test method has exactly one Test.startTest() and one Test.stopTest().
</description>

<priority>1</priority>

<properties>
    <property name="xpath">
        <value>
            <![CDATA[
            //MethodDeclaration[
                (
                    .//Annotation/Name[
                        lower-case(@Image)='istest'
                        or lower-case(@Image)='testmethod'
                    ]
                )
                and
                (
                    count(.//MethodCall[
                        .//Name[lower-case(@Image)='test.starttest']
                    ]) != 1
                    or
                    count(.//MethodCall[
                        .//Name[lower-case(@Image)='test.stoptest']
                    ]) != 1
                )
            ]
            ]]>
        </value>
    </property>
</properties>

I am not able to upload files due to company policy. So added both files directly in description.

Output / Logs

Steps To Reproduce

We ran PMd Scan in our SFOrg. But could not see any Test class where there are Testmethod without Test.startTest(); and Test.stopTest();

Expected Behavior

We are expecting solution to this issue

Operating System

Windows

Salesforce CLI Version

Latest CLI used by Our Development Team

Code Analyzer Plugin (code-analyzer) Version

code-analyzer 5.13.0

Node Version

No response

Java Version

No response

Python Version

No response

Additional Context (Screenshots, Files, etc)

No response

Workaround

No response

Urgency

High

[nikhil-mittal-165]

Hi @mahamuneeprasad,

Thanks for reporting this issue. After investigating, I can confirm that Code Analyzer is working correctly - the issue is with the XPath rule syntax in your custom ruleset.

Root Cause
Your XPath uses node names that don't exist in PMD 7.x's Apex AST. PMD 7.x changed the underlying parser, which means some node names and attributes are different from older versions.

Solution
Replace your XPath with:

//Method[./ModifierNode/Annotation[lower-case(@name)='istest']][count(.//MethodCallExpression[lower-case(@FullMethodName)='test.starttest']) != 1 or count(.//MethodCallExpression[lower-case(@FullMethodName)='test.stoptest']) != 1]

Key changes:

MethodDeclaration → Method
Annotation/Name[@image='istest'] → ModifierNode/Annotation[@name='istest']
MethodCall with Name[@image] → MethodCallExpression[@FullMethodName]

How to Verify
You can use PMD's AST dump command to see the actual node structure for your Apex files:

pmd ast-dump --format xml --language apex --file YourTestClass.cls
This will show you the exact node names and attributes that PMD 7.x uses.

References
PMD 7.x AST Dump Documentation
PMD 7.x Apex AST Node Definitions
Let me know if the corrected XPath works for you!

[mahamuneeprasad]

Hello @nikhil-mittal-165 , Thank you for sharing details on updated XPath. We tried proposed XPath rule in our setup, however still it is not getting triggered during scan. Can we please reopen this Case and provide further assistance.
Result : Rule is not reporting any violation even for test methods that clearly do not contain Test.startTest() and Test.stopTest(). Please see below for same(Ruleset, YML file, Testmethod and PMD Result).
Do you think this is related to Code Analyzer Setup issue in VS Code? Please share if you have any steps. We will follow correct steps.

Ruleset:

<ruleset
name="Apex Design Rules"
xmlns="http://pmd.sourceforge.net/ruleset/2.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 https://pmd.sourceforge.io/ruleset_2_0_0.xsd"

<description>
    Custom Salesforce PMD rules to enforce usage of Test.startTest() and Test.stopTest() in all test methods
</description>

<!-- Rule: Missing Test.startTest() or Test.stopTest() -->
<rule name="TestMethodsMustUseStartStopTest"
  language="apex"
  message="Test methods must contain exactly one Test.startTest() and one Test.stopTest()."
  class="net.sourceforge.pmd.lang.rule.xpath.XPathRule">

<description>
    Ensures every test method has exactly one Test.startTest() and one Test.stopTest().
</description>

<priority>1</priority>

<properties>
    <property name="xpath">
        <value>
            //Method[./ModifierNode/Annotation[lower-case(@name)='istest']][count(.//MethodCallExpression[lower-case(@FullMethodName)='test.starttest']) != 1 or count(.//MethodCallExpression[lower-case(@FullMethodName)='test.stoptest']) != 1]
        </value>
    </property>
</properties>

YML
engines:
flow:
disable_engine: true

pmd:
disable_engine: false
file_extensions:
apex:
- .cls
- .trigger
custom_rulesets:
- rulesets/design/TestMethodsMustUseStartStopTest.xml

TestMethod
@istest
static void testCreateContactsForAccounts() {

    List<Account> accList = [SELECT Id, Name FROM Account];
    System.debug('Error inserting contacts: ');
    AccountContactService.createContactsForAccounts(accList);

    List<Contact> contacts = [
        SELECT Id, LastName, AccountId, Email
        FROM Contact
    ];

    System.assertEquals(
        accList.size(),
        contacts.size(),
        'Number of contacts created should match number of accounts'
    );

    for (Contact con : contacts) {
        System.assertNotEquals(
            null,
            con.AccountId,
            'Contact should be linked to an Account'
        );

        System.assertNotEquals(
            null,
            con.LastName,
            'Contact LastName should not be null'
        );
 }

}

PMD Result
Found 1 violation(s) across 0 file(s):

1 Critical severity violation(s) found.


Violation file paths relative to 'C:\Users\SB126691\Downloads\30June\singlefile'.

Severity Rule Location Message

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

1 3 (Moderate) pmd:MissingEncoding rulesets\design\TestMethodsMustUseStartStopTest.xml:1:1 Set an explicit XML encoding in the XML declaration to ensure proper parsing

2 4 (Low) pmd:ApexDoc force-app\main\default\classes\AccountContactService.cls:1:21 Missing ApexDoc comment

3 4 (Low) pmd:ApexDoc force-app\main\default\classes\AccountContactService.cls:3:19 Missing ApexDoc comment

4 4 (Low) pmd:AvoidDebugStatements force-app\main\default\classes\AccountContactService.cls:14:17 Avoid debug statements since they impact on performance

5 4 (Low) pmd:DebugsShouldUseLoggingLevel force-app\main\default\classes\AccountContactService.cls:14:17 Calls to System.debug should specify a logging level.

6 4 (Low) pmd:AvoidDebugStatements force-app\main\default\classes\AccountContactService.cls:35:16 Avoid debug statements since they impact on performance

7 4 (Low) pmd:DebugsShouldUseLoggingLevel force-app\main\default\classes\AccountContactService.cls:35:16 Calls to System.debug should specify a logging level.

8 4 (Low) pmd:AnnotationsNamingConventions force-app\main\default\classes\AccountContactServiceTest.cls:1:1 The annotation @istest should be in PascalCase: @istest

9 4 (Low) pmd:AnnotationsNamingConventions force-app\main\default\classes\AccountContactServiceTest.cls:4:5 The annotation @testsetup should be in PascalCase: @testsetup

10 4 (Low) pmd:AvoidDebugStatements force-app\main\default\classes\AccountContactServiceTest.cls:13:9 Avoid debug statements since they impact on performance

11 4 (Low) pmd:DebugsShouldUseLoggingLevel force-app\main\default\classes\AccountContactServiceTest.cls:13:9 Calls to System.debug should specify a logging level.

12 4 (Low) pmd:AnnotationsNamingConventions force-app\main\default\classes\AccountContactServiceTest.cls:17:5 The annotation @istest should be in PascalCase: @istest

13 4 (Low) pmd:ApexUnitTestClassShouldHaveRunAs force-app\main\default\classes\AccountContactServiceTest.cls:18:12 Apex unit test classes should have at least one System.runAs() call

14 4 (Low) pmd:AvoidDebugStatements force-app\main\default\classes\AccountContactServiceTest.cls:21:9 Avoid debug statements since they impact on performance

15 4 (Low) pmd:DebugsShouldUseLoggingLevel force-app\main\default\classes\AccountContactServiceTest.cls:21:9 Calls to System.debug should specify a logging level.

16 4 (Low) pmd:AnnotationsNamingConventions force-app\main\default\classes\AccountContactServiceTest.cls:50:5 The annotation @istest should be in PascalCase: @istest

17 4 (Low) pmd:ApexUnitTestClassShouldHaveRunAs force-app\main\default\classes\AccountContactServiceTest.cls:51:12 Apex unit test classes should have at least one System.runAs() call

=== Summary

Found 17 violation(s) across 3 file(s):

1 Moderate severity violation(s) found.

16 Low severity violation(s) found.

[nikhil-mittal-165]

Hi @mahamuneeprasad,

Thanks for the follow-up! I think I see the issue — there's a small typo in the XPath you used.

In your updated ruleset, you have:

//Method[./ModifierNode/Annotation[lower-case(@name)='istest']]...

But it should be @Name (capital N), not @name:

//Method[./ModifierNode/Annotation[lower-case(@Name)='istest']][count(.//MethodCallExpression[lower-case(@FullMethodName)='test.starttest']) != 1 or count(.//MethodCallExpression[lower-case(@FullMethodName)='test.stoptest']) != 1]


XPath attribute names are case-sensitive, so @name doesn't match anything in PMD's AST and the rule silently returns zero results.

Debugging tip for the future: Use PMD's AST dump to inspect the exact node/attribute names for your Apex files:

pmd ast-dump --format xml --language apex --file YourTestClass.cls


This will let you write XPath expressions with confidence for any future custom rules.

Also a minor note: the MissingEncoding violation on your ruleset XML means PMD is scanning the ruleset file itself. You can either exclude that directory from your scan targets, or add an encoding declaration:


<?xml version="1.0" encoding="UTF-8" ?>


I don't think its a code analyzer issue and is a xpath syntax issue on your file as the code analyzer loaded the ruleset, ran PMD, and scanned the files. 
Reopening the issue while you work through this

[mahamuneeprasad]

@nikhil-mittal-165 Thank you so much for reopening Case and your valuable suggestion. I have forwarded these details to my development team, we will get back to you with updates ASAP.