| Age | Commit message (Collapse) | Author |
|---|
|
Even when the kernel panics and qemu dies, runs with jitter enabled will
continue uselessly until the jitter.sh processes terminate. This can
be annoying if a planned one-hour run instead dies during boot.
This commit therefore kills the jitter.sh processes when the run ends
more than one minute prior to the termination time specified by the
kvm.sh --duration argument or its default.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
The rcutorture scripting will do a "kill -9" on any guest OS that exceeds
its --duration by more than a few minutes, which is very valuable when
bugs result in hangs. However, this is a problem when the "hang" was due
to a --gdb debugging session.
This commit therefore refrains from killing the guest OS when a debugging
session is in progress. This means that the user must manually kill the
kvm.sh process group if a hang really does occur.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
This commit adds a "--gdb" parameter to kvm.sh, which causes
"CONFIG_DEBUG_INFO=y" to be added to the Kconfig options, "nokaslr"
to be added to the boot parameters, and "-s -S" to be added to the qemu
arguments. Furthermore, the scripting prints messages telling the user
how to start up gdb for the run in question.
Because of the interactive nature of gdb sessions, only one "--configs"
scenario is permitted when "--gdb" is specified. For most torture types,
this means that a "--configs" argument is required, and that argument
must specify the single scenario of interest.
The usual cautions about breakpoints and timing apply, for example,
staring at your gdb prompt for too long will likely get you many
complaints, including RCU CPU stall warnings. Omar Sandoval further
suggests using gdb's "hbreak" command instead of the "break" command on
systems supporting hardware breakpoints, and further using the "commands"
option because the resulting non-interactive breakpoints are less likely
to get you RCU CPU stall warnings.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
In the dim distant past, qemu commands needed to be run from the
rcutorture directory, but this is no longer the case. This commit
therefore removes the now-useless "cd $KVM" from the kvm-test-1-run.sh
script.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
Currently, the qemu command is constructed twice, once to dump it
to the qemu-cmd file and again to execute it. This is of course an
accident waiting to happen, but is done to ensure that the remainder
of the script has an accurate idea of the running qemu command's PID.
This commit therefore places both the qemu command and the PID capture
into a new temporary file and sources that temporary file. Thus the
single construction of the qemu command into the qemu-cmd file suffices
for both purposes.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
When bisecting RCU issues, it is often the case that the first error in
an unsuccessful run will happen quickly, but that a successful run must
go on for some time in order to obtain a sufficiently low false-negative
error rate. In many cases, a bisection requires multiple concurrent
runs, in which case the first failure in any run indicates failure,
pure and simple. In such cases, it would speed things up greatly if
the first failure terminated all runs.
This commit therefore adds scripting that checks for a file named "STOP"
in the top-level results directory, terminating the run when it appears.
Note that in-progress builds will continue until completion, but future
builds and all runs will be cut short.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
One reason to do a --buildonly run is to use the build products elsewhere,
for example, to do the actual test on some other system. Part of doing
the test is the actual qemu command, which is not currently produced
by --buildonly runs. This commit therefore causes --buildonly runs to
create this file.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
On some (probably misconfigured) systems, the torture-test scripting
will cause qemu to complain about missing EFI firmware, often because
qemu is trying to traverse broken symbolic links to find that firmware.
Which is a bit silly given that the default torture-test guest OS has
but a single binary for its userspace, and thus is unlikely to do much
in the way of networking in any case.
This commit therefore avoids such problems by specifying "-net none"
to qemu unless the TORTURE_QEMU_INTERACTIVE environment variable is set
(for example, by having specified "--interactive" to kvm.sh), in which
case "-net nic -net user" is specified to qemu instead. Either choice
may be overridden by specifying the "-net" argument of your choice to
the kvm.sh "--qemu-args" parameter.
Link: https://lore.kernel.org/lkml/20190701141403.GA246562@google.com
Reported-by: Joel Fernandes <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
|
|
Make it a bit easier to apply KASAN to rcutorture runs with a new --kasan
argument, again leveraging the config_override_param() bash function.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
This commit saves a few lines of code by also using the bash
config_override_param() to set the initial list of Kconfig options from
the CFcommon file. While in the area, it makes this function capable of
update-in-place on the file containing the cumulative Kconfig options,
thus avoiding annoying changes when adding another source of options.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
This commit applies config_override_param() to allow scenario-specific
Kconfig options to override those in CFcommon. This in turn will allow
additional Kconfig options to be placed in CFcommon, for example, an
option common to all but a few scenario can be placed in CFcommon and
then overridden in those few scenarios. Plus this change saves one
whole line of code.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
Currently, attempting to override a --kcsan default with a --kconfig
option might or might not work. However, it would be good to allow the
user to adjust the --kcsan defaults, for example, to specify a different
time for CONFIG_KCSAN_REPORT_ONCE_IN_MS. This commit therefore uses the
new config_override_param() bash function to apply the --kcsan defaults
and then apply the --kconfig options, which allows this overriding
to occur.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
This commit introduces a config_override_param() bash function that
folds in an additional set of Kconfig options. This is initially applied
to fold in the --kconfig kvm.sh parameter, but later commits will also
apply it to the Kconfig options added by the --kcsan kvm.sh parameter.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
The #CHECK# directives that can be present in CFcommon and in the
rcutorture scenario Kconfig files are both copied to ConfigFragment
and grepped out of the two directive files and added to ConfigFragment.
This commit therefore removes the redundant "grep" commands and takes
advantage of the consequent opportunity to simplify redirection.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
On some kernels, concurrent calls to the lscpu command result in severe
slowdowns. For example, on v4.16, a single lscpu invocation takes about
two milliseconds, four concurrent invocations more than two seconds,
and 16 concurrent invocations more than 20 seconds. Given that the only
goal is to learn the number of CPUs, invoking lscpu but once suffices.
This commit therefore invokes lscpu early in kvm.sh execution, setting
the initial value of the TORTURE_ALLOTED_CPUS environment variable.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
In many environments, gawk provides systime(), but awk doesn't.
This commit therefore changes awk scripts using systime() to instead be
gawk scripts.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
The kvm-test-1-run.sh script says 'test -z "last_ts"' which always
evaluates to true (AKA zero) regardless of the value of the last_ts shell
variable. This commit therefore inserts the needed dollar sign ("$").
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
|
|
For historical reasons, rcutorture places its build products in a
tools/testing/selftests/rcutorture/b1 directory using the O= kbuild
command-line argument. However, doing this requires that the source
directory be pristine: Not just "make clean" pristine, but instead "make
mrproper" (or, equivalently, "make distclean") pristine. Therefore,
rcutorture executes a "make mrproper" before each build. Unfortunately,
"make mrproper" has the side effect of removing pretty much everything,
including tags files and cscope databases, which can be inconvenient
to people whose workflow centers around a single source tree.
This commit therefore makes rcutorture do the build directly in the
source directory, removing the need for "make mrproper". This works
because all needed build products are moved to their proper place in the
"res" directory immediately after the build completes, so that multiple
rcutorture kernels can still run concurrently.
Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
|
|
Currently qemu output appears on standard output, but is inaccessible
later on. This commit therefore captures this output and causes
kvm-recheck.sh to output this output if QEMU gave a non-zero non-137
exit code. (And exit code of 137 indicates that QEMU was killed, in
which case we want to know about the hang rather than the fact that
QEMU was killed.)
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
|
|
Replace the license boiler plate with a SPDX license identifier.
While in the area, update an email address and add copyright notices.
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
|
|
The old approach placed all the build products into the b* directories,
which meant that some of these build products needed to be copied to
the proper directory in the res hierarchy. The new approach leaves
things like .config and the .o files in the b1 directory, but directs
build output and diagnostics directly to the proper directory in the
res hierarchy. Unfortunately, one of the copies was still carried out,
which could (and sometimes did) overwrite the build output and diagnostics
with obsolete output remaining in the b1 directory.
This commit therefore removes the offending "cp" command.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
The torture scripting currently builds each kernel from a set of parallel
runs in its own build directory. This can waste quite a bit of space when
running large numbers of concurrent scenarios, and pointlessly given that
the builds are run sequentially (albeit with a largish -j number passed to
"make"). This commit therefore places all build-command output in the
results directory, then does all builds in a single "b1" build directory.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
The rcutorture scripting scans the console output twice, once to look
for various sorts of hangs and again to find warnings and panics.
Unfortunately, only the output of the second scan gets written to the
console.log.diags file, which can cause hangs to be overlooked.
This commit therefore folds the parse-torture.sh script (which looks
for hangs) into the parse-console.sh script (which looks for warnings
and panics). This allows both types of failure information to be
added to console.log.diags, while still reliably removing this file
when it proves to be empty.
This also fixes a long-standing bug where rcuperf log files would
unconditionally complain about a hang.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>
|
|
The 512 megabyte memory size has served quite well, but more memory
is required when using large trace buffers on large systems. This
commit therefore adds a --memory argument to the kvm.sh script, which
allows the memory size to be specified on the command line, for example,
"--memory 768", --memory 800M", or "--memory 2G".
Reported-by: Sebastian Siewior <bigeasy@linutronix.de>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
Inclusions of 'functions.sh' from 'kvm-test-1-run.sh' and
'kvm-recheck*.sh' use its absolute path. Because the directory containing
'functions.sh' is already in PATH, the full path is unnecessary. This
commit therefore simplifies the inclusions to use the short relative path.
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
Both the 'kvm.sh' and 'kvm-test-1-run.sh' scripts log messages by printing
the message to 'stdout' and then also printing it into the log file.
Generation of the message thus occurs twice, once for 'stdout' and once
for the log file. Moreover, many of the messages contain 'date' output,
which results in date being invoked twice (once for stdout print, once
for log file write). As a result, the date information in stdout and
log file can differ, which could cause confusion.
This commit therefore simplifies the logging procedure by using 'tee'.
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
Both rcutorture and locktorture currently place temporary files in /tmp,
in keeping with decades-long tradition. However, sometimes it is useful
to specify an alternative temporary directory, for example, for space
or performance reasons. This commit therefore causes the torture-test
scripting to use the path specified in the TMPDIR environment variable,
or to fall back to traditional /tmp if this variable is not set.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
Currently, rcutorture groups runs in batches, building each scenario in
a given batch, then invoking qemu to run all the kernels in the batch.
Of course, if a given scenario's kernel fails to build, there is no qemu
run for that scenario. And if all of the kernels in a given batch fail
to build, there are no runs, and rcutorture immediately starts on the
next batch.
But not if --jitter has been specified, which it is by default. In this
case, the jitter scripts are started unconditionally, and rcutorture
waits for them to complete, even though there are no kernels to run.
This commit therefore checks for this situation, and refuses to start
jitter unless at least one of the kernels in the batch built successfully.
This saves substantial time when all scenarios' kernels fail to build,
particularly if a long --duration was specified.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
Currently, testing a variant of an existing scenario requires editing
that scenario's file or creating a new scenario file. This is messy
and error prone with respect to changes to scenarios.
This commit therefore adds a --kconfig argument to kvm.sh, so that
'--kconfig "CONFIG_RCU_TRACE=y CONFIG_RCU_EQS_DEBUG=n" will override those
two Kconfig options. In addition, there is now clear precedence:
the config fragment overrides CFcommon, and the --kconfig argument
overrides both.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
Currently, the rcutorture scripting will give an error message if
running a duplicate scenario that happens also to have a non-existent
build directory (b1, b2, ... in the rcutorture directory). Worse yet, if
the build directory has already been created and used for a real build,
the script will silently grab the wrong Kconfig fragment, which could
cause confusion to the poor sap (me) analyzing old test results. At
least the actual test runs correctly...
This commit therefore accesses the Kconfig fragment from the results
directory corresponding to the first of the duplicate scenarios, for
which a build was actually carried out. This prevents both the messages
and at least one form of later confusion.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
The option "-soundhw pcspk" gives me a error on PPC as follow:
qemu-system-ppc64: ISA bus not available for pcspk
This means this option doesn't work on ppc by default. So simply make
this an x86-specific option via identify_qemu_args().
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
The vmlinux image is available for all the architectures, and suitable
for running a KVM guest by QEMU, besides, we used to copy the vmlinux
to $resdir anyway. Therefore it makes sense to use it as the fallback
kernel image for rcutorture KVM tests.
This patch makes identify_boot_image() return vmlinux if
${TORTURE_BOOT_IMAGE} is not set on non-x86 architectures, also fixes
several places that hard-code "bzImage" as $KERNEL.
This also fixes a problem that PPC doesn't have a bzImage file as build
results.
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
This commit avoids killing qemu if a trace dump is making progress
or if console log output is continuing and the console log timestamp
does not exceed the total plus grace period.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
Currently, if the user specifies multiple runs of a given test
configuration, the scripting does multiple kernel builds. This wastes
both time and disk space, so this commit makes the scripting use the
first build for all runs of a given test configuration.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
This commit converts test duration from minutes to seconds early on
in order to prepare for upcoming OS-jitter-injection changes.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
The current hang-check machinery in the rcutorture scripts uses "$!" of
a parenthesized bash statement to capture the pid. Unfortunately, this
captures not qemu's pid, but rather that of its parent that implements
the parenthesized statement. This commit therefore adjusts things so as
to capture qemu's actual pid, which then allows the script to actually
kill qemu in event of a kernel hang.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
Currently, the console output files ("console.log") are placed in the
build directory initially, then copied to the results directory.
One problem with this is if a qemu refuses to die in a timely fashion
after a kernel hang, it will continue to write after the next qemu
starts up, resulting in confusing output from the old instance of
qemu. This commit prevents such confusion by placing the console.log
files into the results directory to begin with, so that a given instance
of qemu is always writing only to its own console.log file.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
|
|
The default test grace period of two minutes is insufficient in some
cases and excessive in others. This commit therefore increases the
default to three minutes, but also adds a --shutdown-grace parameter
to allow the default to be overridden.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
|
|
A system misconfiguration that prevents qemu from running at all (for
example, a missing dynamically linked library) will keep the console.log
file from the previous run. This can fool the developer into thinking
that this failed run actually completed correctly. This commit therefore
overwrites the console.log file just before launching qemu.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
More recent qemu implementations really want "-enable-kvm", and the
"-soundhw pcspk" makes the script a bit less dependent on odd audio
libraries being installed. This commit therefore adds both to the
default qemu command line.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
Now that we have removed configs based on kernel version, we can also remove the
kversion parameter in kvm.sh.
Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
The rcutorture scripts create a qemu-cmd script containing the actual
qemu command. However, this command references the build directory,
which will be overwritten by later builds. This commit therefore runs
the kernel out of the results directory so that less hand-editing is
required to re-run a previous test.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Pranith Kumar <bobby.prani@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
|
|
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Pranith Kumar <bobby.prani@gmail.com>
|
|
Setting CONFIG_MAXSMP=y causes cpumasks to be moved offstack, which
introduces the possibility of NULL cpumask_var_t pointers. This commit
therefore enables CONFIG_MAXSMP=y in TREE01 to increase test coverage.
However, because CONFIG_MAXSMP=y implies 8192 CPUs, we need to use
the maxcpus= boot parameter to limit the number of CPUs to something
reasonable, which in turn requires updating the scripts to handle this.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Pranith Kumar <bobby.prani@gmail.com>
|
|
This commit sets the executable bit on test scripts config2frag.sh
and kvm.sh. Since #!/bin/bash is set in all the scripts, this commit
also drops it from all usage lines because the scripts can now all be
invoked directly.
Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
Some of the scripts encode a default /bin/sh shell. On systems which use
dash as default shell, these scripts fail as they are bash scripts. I
encountered this while testing the sprintf() changes on a Debian system
where dash is the default shell.
This commit changes all such uses to use bash explicitly.
Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
Currently the post-processing complains about the lack of rcutorture
output when --buildonly is set and also emits misleading messages about
kernels being started and finishing. This commit suppresses these
complaints and messages.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
The CFcommon file must now be present, which makes using the current
scripts against old kernel versions cumbersome. This commit therefore
makes the CFcommon file be optional, so that old kernel versions can be
used with current torture scripts.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
|
The kvm-test-1-run.sh currently counts "sleep 1" commands to detect
hangs. This can fail spectacularly on busy systems, where "sleep 1"
might take far longer than one second to complete. This commit
therefore changes hang detection to use elapsed time measurements.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
|
|
The reaction of kvm-recheck.sh is obscure at best, and easy to miss
completely. This commit therefore prints "BUG: Build failed" in the
summary at the end of a run. This commit also adds the line of dashes
in cases where performance info is not available, and also avoids
printing nonsense diagnostics in cases where some of the normal test
output is not available. In addition, this commit saves off the .config
file even when the build fails.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
|
