Test Tools
SG-Runner is a modular framework designed to integrate with a wide range of tools. Using Jinja2 templates, provids dynamic and configurable tool definitions. Templates allow users to parameterize commands, configuration files, and workflows using variables, conditionals, and reusable components. SG-Runner Tools is capable of transforming reports into graphical representations and generating statistical analysis results.
Supported test tools:
- Seagull: Multi-protocol 4G/5G traffic symulator
- Jmeter: SOAP/XML/REST Webservices traffic generator
Seagull Tool
Seagull is a free, Open Source (GPL) multi-protocol traffic generator test tool. Primarily aimed at IMS (3GPP, TISPAN, CableLabs) protocols (and thus being the perfect complement to SIPp for IMS testing), Seagull is a powerful traffic generator for functional, load, endurance, stress and performance/benchmark tests for almost any kind of protocol.
Seagull Features:
Seagull has the following features:
- Multi-protocol traffic generator
- Command line tool with text interface
- Protocols of the same family are described in an XML, user editable, dictionary (messages, parameters)
- Existing protocol families: Binary/TLV (Type, Length, Value), Raw binary, Text, external API (first implementation: HP OpenCall SS7)
- Support of IP (UDP/TCP), SCTP, SSL/TLS and SS7/TCAP transports
- Portable programming (tested and supported on Linux x86, ia64, HPUX, SunOS and Windows)
- Scenarios are described using XML files
- Multi-threaded for performances and reliability
- Dynamically adjustable scenario rate
- Uniform, Poisson or Best-effort scenario arrival distribution
- Remote-control (scenario-rate set, counter dump) through standard HTTP interface
- Pause and restart of traffic
- Support of automated traffic profile (varying scenario rate)
- Smooth (no new scenarios then wait for ongoing scenarios to end) or brutal end
- Scenario display with message counters
- Scenarios have init (executed once), main (repeated for traffic) sections
- Scenarios have default sections for defense in case of unexpected messages
- A scenario can be mono (most cases) or multi-protocol
- Message and parameters checking possible (disabled by default)
- Support of parameter injection following a CSV like database
- Multiple Seagull instances can be synchronized in the middle of scenario
- Intra scenario synchronization using a synchronization protocol (example application provided in Java language)
- Statistics: timer between two messages, scenario length, scenario rate, successful scenarios, failed scenarios (with reason)
- Protocol decoding and hexadecimal dump
- Trace files with or without timestamps (for performances and automation)
JMeter
Apache JMeter may be used to test performance both on static and dynamic resources, Web dynamic applications. It can be used to simulate a heavy load on a server, group of servers, network or object to test its strength or to analyze overall performance under different load types.
JMeter Features:
Apache JMeter features include:
- Ability to load and performance test many different applications/server/protocol types:
- Web - HTTP, HTTPS (Java, NodeJS, PHP, ASP.NET, …)
- SOAP / REST Webservices
- FTP
- Database via JDBC
- LDAP
- Message-oriented middleware (MOM) via JMS
- Mail - SMTP(S), POP3(S) and IMAP(S)
- Native commands or shell scripts
- TCP
- Java Objects
- Full featured Test IDE that allows fast Test Plan recording (from Browsers or native applications), building and debugging.
- CLI mode (Command-line mode (previously called Non GUI) / headless mode) to load test from any Java compatible OS (Linux, Windows, Mac OSX, …)
- A complete and ready to present dynamic HTML report
- Easy correlation through ability to extract data from most popular response formats, HTML, JSON , XML or any textual format
- Complete portability and 100% Java purity.
- Full multi-threading framework allows concurrent sampling by many threads and simultaneous sampling of different functions by separate thread groups.
- Caching and offline analysis/replaying of test results.
- Highly Extensible core:
- Pluggable Samplers allow unlimited testing capabilities.
- Scriptable Samplers (JSR223-compatible languages like Groovy and BeanShell)
- Several load statistics may be chosen with pluggable timers.
- Data analysis and visualization plugins allow great extensibility as well as personalization.
- Functions can be used to provide dynamic input to a test or provide data manipulation.
- Easy Continuous Integration through 3rd party Open Source libraries for Maven, Gradle and Jenkins.