Hacking Articles Tips Tricks Videos Tutorials

REST API fuzzer and negative testing tool. Run thousands of self-healing API tests within minutes with no coding effort!Comprehensive: tests are generated automatically based on a large number scenarios and cover every field and headerIntelligent: tests are generated based on data types and constraints; each Fuzzer have specific expectations depending on the scenario under testHighly Configurable: high amount of customization: you can exclude specific Fuzzers, HTTP response codes, provide business context and a lot moreSelf-Healing: as tests are generated, any OpenAPI spec change is picked up automaticallySimple to Learn: flat learning curve, with intuitive configuration and syntaxFast: automatic process for write, run and report tests which covers thousands of scenarios within minutes
OverviewBy using a simple and minimal syntax, with a flat learning curve, CATS (Contract Auto-generated Tests for Swagger) enables you to generate thousands of API tests within minutes with no coding effort. All tests are generated, run and reported automatically based on a pre-defined set of 89 Fuzzers. The Fuzzers cover a wide range of input data from fully random large Unicode values to well crafted, context dependant values based on the request data types and constraints. Even more, you can leverage the fact that CATS generates request payloads dynamically and write simple end-to-end functional tests.

___________________________
@hacking_Attack
@Hacking_Video

6 viewsedited 13:34

Please check the Slicing Strategies (https://github.com/Endava/cats#slicing-strategies-for-running-cats) section for making CATS run fast and comprehensive in the same time. Tutorials on how to use CATSThis is a list of articles with step-by-step guides on how to use CATS:Testing the GitHub API with CATS (https://ludovicianul.github.io/2020/10/05/github-api-testing/)How to write self-healing functional tests with no coding effort (https://ludovicianul.github.io/2020/09/09/cats/)Some bugs found by CATShashicorp/vault#13274 (https://github.com/hashicorp/vault/issues/13274) | hashicorp/vault#13273 (https://github.com/hashicorp/vault/issues/13273)hashicorp/vault#13225 (https://github.com/hashicorp/vault/issues/13225) | hashicorp/vault#13232 (https://github.com/hashicorp/vault/issues/13232)go-gitea/gitea#19397 (https://github.com/go-gitea/gitea/issues/19397) | go-gitea/gitea#19398 (https://github.com/go-gitea/gitea/issues/19398)go-gitea/gitea#19399 (https://github.com/go-gitea/gitea/issues/19399)InstallationHomebrew brew tap endava/tap > brew install cats">> brew tap endava/tap
> brew install catsManualCATS is bundled both as an executable JAR or a native binary. The native binaries do not need Java installed.After downloading your OS native binary, you can add it in classpath so that you can execute it as any other command line (https://www.kitploit.com/search/label/Command%20Line) tool:sudo cp cats /usr/local/bin/catsYou can also get autocomplete by downloading the cats_autocomplete (https://github.com/Endava/cats/blob/master/cats_autocomplete) script and do:source cats_autocompleteTo get persistent autocomplete, add the above line in ~/.zshrc or ./bashrc, but make sure you put the fully qualified path for the cats_autocomplete script.You can also check the cats_autocomplete source for alternative setup.There is no native binary for Windows, but you can use the uberjar version. This requires Java 11+ to be installed.You can run it as java -jar cats.jar.Head to the releases page to download the latest versions: https://github.com/Endava/cats/releases.BuildYou can build CATS from sources on you local box. You need Java 11+. Maven is already bundled.Before running the first build, please make sure you do a ./mvnw clean. CATS uses a fork ok OKHttpClient which will install locally under the 4.9.1-CATS version, so don't worry about overriding the official versions.You can use the following Maven command to build the project:./mvnw package -Dquarkus.package.type=uber-jarcp target/You will end up with a cats.jar in the target folder. You can run it wih java -jar cats.jar ....You can also build native images using a GraalVM Java version../mvnw package -PnativeNote: You will need to configure Maven with a Github PAT (https://docs.github.com/en/free-pro-team@latest/packages/guides/configuring-apache-maven-for-use-with-github-packages) with read-packages scope to get some dependencies for the build.Notes on Unit TestsYou may see some ERROR log messages while running the Unit Tests. Those are expected behaviour for testing the negative scenarios of the Fuzzers.Running CATSBlackbox modeBlackbox mode means that CATS doesn't need any specific context. You just need to provide the service URL, the OpenAPI spec and most probably authentication headers (https://github.com/Endava/cats#headers-file).> cats --contract=openapy.yaml --server=http://localhost:8080 --headers=headers.yml --blackboxIn blackbox mode CATS will only report ERRORs if the received HTTP response code is a 5XX. Any other mismatch between what the Fuzzer expects vs what the service returns (for example service returns 400 and service returns 200) will be ignored.The blackbox mode is similar to a smoke test. It will quickly tell you if the application has major bugs that must be addressed immediately.Context modeThe real power of CATS relies on running it in a non-blackbox mode also called context mode. Each Fuzzer has an expected HTTP response code based on the scenario under

___________________________
@hacking_Attack
@Hacking_Video

GitHub

GitHub - Endava/cats: CATS is a REST API Fuzzer and negative testing tool for OpenAPI endpoints. CATS automatically generates…

CATS is a REST API Fuzzer and negative testing tool for OpenAPI endpoints. CATS automatically generates, runs and reports tests with minimum configuration and no coding effort. Tests are self-heal...

6 viewsedited 13:34

Hacking Articles Tips Tricks Videos Tutorials

test and will also check if the response is matching the schema defined in the OpenAPI spec specific to that response code. This will allow you to tweak either your OpenAPI spec or service behaviour in order to create good quality APIs and documentation and also to avoid possible serious bugs.Running CATS in context mode usually implies providing it a --refData (https://github.com/Endava/cats#reference-data-file) file with resource identifiers specific to the business logic. CATS cannot create data on its own (yet), so it's important that any request field or query param that requires pre-existence of those entities/resources to be created in advance and added to the reference data file.> cats --contract=openapy.yaml --server=http://localhost:8080 --headers=headers.yml --refData=referenceData.ymlNotes on skipped TestsYou may notice a significant number of tests marked as skipped. CATS will try to apply all Fuzzers to all fields, but this is not always possible. For example the BooleanFieldsFuzzer cannot be applied to String fields. This is why that test attempt will be marked as skipped. It was an intentional decision to also report the skipped tests in order to show that CATS actually tries all the Fuzzers on all the fields/paths/endpoints.Additionally, CATS support a lot more arguments (https://github.com/Endava/cats#available-arguments) that allows you to restrict the number of fuzzers, provide timeouts, limit the number of requests per minute and so on.Understanding how CATS works and reports resultsCATS generates tests based on configured Fuzzers. Each Fuzzer has a specific scenario and a specific expected result. The CATS engine will run the scenario, get the result from the service and match it with the Fuzzer expected result. Depending on the matching outcome, CATS will report as follows:INFO/SUCCESS is expected and documented behaviour. No need for action.WARN is expected but undocumented behaviour or some misalignment between the contract and the service. This will ideally be actioned.ERROR is abnormal/unexpected behaviour. This must be actioned.CATS will iterate through all endpoints, all HTTP methods and all the associated requests bodies and parameters (including multiple combinations when dealing with oneOf/anyOf elements) and fuzz their values considering their defined data type and constraints. The actual fuzzing depends on the specific Fuzzer executed. Please see the list of fuzzers and their behaviour. There are also differences on how the fuzzing works depending on the HTTP method:for methods with request bodies like POST, PUT the fuzzing will be applied at the request body data models levelfor methods without request bodies like GET, DELETE the fuzzing will be applied at the URL parameters levelThis means that for methods with request bodies (POST,PUT) that have also URL/path parameters, you need to supply the path parameters via urlParams or the referenceData file as failure to do so will result in Illegal character in path at index ... errors.Interpreting ResultsHTML_JSHTML_JS is the default report produced by CATS. The execution report in placed a folder called cats-report/TIMESTAMP or cats-report depending on the --timestampReports argument. The folder will be created inside the current folder (if it doesn't exist) and for each run a new subfolder will be created with the TIMESTAMP value when the run started. This allows you to have a history of the runs. The report itself is in the index.html file, where you can:filter test runs based on the result: All, Success, Warn and Errorfilter based on the Fuzzer so that you can only see the runs for that specific Fuzzersee summary with all the tests with their corresponding path against they were run, and the resulthave ability to click on any tests and get details about the Scenario being executed, Expected Result, Actual result as well as request/response detailsAlong with the summary from index.html each individual test will have a specific TestXXX.html

___________________________
@hacking_Attack
@Hacking_Video

GitHub

GitHub - Endava/cats: CATS is a REST API Fuzzer and negative testing tool for OpenAPI endpoints. CATS automatically generates…

CATS is a REST API Fuzzer and negative testing tool for OpenAPI endpoints. CATS automatically generates, runs and reports tests with minimum configuration and no coding effort. Tests are self-heal...

7 viewsedited 13:34

Hacking Articles Tips Tricks Videos Tutorials

page with more details, as well as a json version of the test which can be latter replayed using > cats replay TestXXX.json.Understanding the Result Reason values:Unexpected Exception - reported as error; this might indicate a possible bug in the service or a corner case that is not handled correctly by CATSNot Matching Response Schema - reported as a warn; this indicates that the service returns an expected response code and a response body, but the response body does not match the schema defined in the contractUndocumented Response Code - reported as a warn; this indicates that the service returns an expected response code, but the response code is not documented in the contractUnexpected Response Code - reported as an error; this indicates a possible bug in the service - the response code is documented, but is not expected for this scenarioUnexpected Behaviour - reported as an error; this indicates a possible bug in the service - the response code is neither documented nor expected for this scenarioNot Found - reported as an error in order to force providing more context; this indicates that CATS needs additional business context in order to run successfully - you can do this using the --refData and/or --urlParams argumentsThis is the summary page:

___________________________
@hacking_Attack
@Hacking_Video

6 viewsedited 13:34

Hacking Articles Tips Tricks Videos Tutorials

6 views13:34

Hacking Articles Tips Tricks Videos Tutorials

And this is what you get when you click on a specific test:

___________________________
@hacking_Attack
@Hacking_Video

6 viewsedited 13:34

Hacking Articles Tips Tricks Videos Tutorials

6 views13:34

About

Blog

Apps

Platform