Commit Graph

1396 Commits

Author SHA1 Message Date
Markus Armbruster
c9efc984ca qapi: Reorder check_FOO() parameters for consistency
Most check_FOO() take the thing being checked as first argument.
check_name(), check_type(), check_known_keys() don't.  Clean that up.

While there, drop a "Todo" comment that should have been dropped in
commit 87adbbffd4 "qapi: add a dictionary form for TYPE".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-9-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster
481a6bd15c qapi: Improve reporting of member name clashes
We report name clashes like this:

    struct-base-clash.json: In struct 'Sub':
    struct-base-clash.json:5: 'name' (member of Sub) collides with 'name' (member of Base)

The "(member of Sub)" is redundant with "In struct 'Sub'".  Comes from
QAPISchemaMember.describe().  Pass info to it, so it can detect the
redundancy and avoid it.  Result:

    struct-base-clash.json: In struct 'Sub':
    struct-base-clash.json:5: member 'name' collides with member 'name' of type 'Base'

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-8-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster
2ab218aad6 qapi: Change frontend error messages to start with lower case
Starting error messages with a capital letter complicates things when
text can get interpolated both at the beginning and in the middle of
an error message.  The next patch will do that.  Switch to lower case
to keep it simpler.

For what it's worth, the GNU Coding Standards advise the message
"should not begin with a capital letter when it follows a program name
and/or file name, because that isn’t the beginning of a sentence. (The
sentence conceptually starts at the beginning of the line.)"

While there, avoid breaking lines containing multiple arguments in the
middle of an argument.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-7-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster
638c4af931 qapi: Clean up member name case checking
QAPISchemaMember.check_clash() checks for member names that map to the
same c_name().  Takes care of rejecting duplicate names.

It also checks a naming rule: no uppercase in member names.  That's a
rather odd place to do it.  Enforcing naming rules is
check_name_str()'s job.

qapi-code-gen.txt specifies the name case rule applies to the name as
it appears in the schema.  check_clash() checks c_name(name) instead.
No difference, as c_name() leaves alone case, but unclean.

Move the name case check into check_name_str(), less the c_name().
New argument @permit_upper suppresses it.  Pass permit_upper=True for
definitions (which are not members), and when the member's owner is
whitelisted with pragma name-case-whitelist.

Bonus: name-case-whitelist now applies to a union's inline base, too.
Update qapi/qapi-schema.json pragma to whitelist union CpuInfo instead
of CpuInfo's implicit base type's name q_obj_CpuInfo-base.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-6-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster
7be6c51194 qapi: Prefix frontend errors with an "in definition" line
We take pains to include the offending expression in error messages,
e.g.

    tests/qapi-schema/alternate-any.json:2: alternate 'Alt' member 'one' cannot use type 'any'

But not always:

    tests/qapi-schema/enum-if-invalid.json:2: 'if' condition must be a string or a list of strings

Instead of improving them one by one, report the offending expression
whenever it is known, like this:

    tests/qapi-schema/enum-if-invalid.json: In enum 'TestIfEnum':
    tests/qapi-schema/enum-if-invalid.json:2: 'if' condition must be a string or a list of strings

Error messages that mention the offending expression become a bit
redundant, e.g.

    tests/qapi-schema/alternate-any.json: In alternate 'Alt':
    tests/qapi-schema/alternate-any.json:2: alternate 'Alt' member 'one' cannot use type 'any'

I'll take care of that later in this series.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-5-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster
19e950d9d4 qapi: New QAPISourceInfo, replacing dict
We track source locations with a dict of the form

    {'file': FNAME, 'line': LINENO, 'parent': PARENT}

where PARENT is None for the main file, and the include directive's
source location for included files.

This is serviceable enough, but the next commit will add information,
and that's going to come out cleaner if we turn this into a class.  So
do that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-4-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster
57608a5299 qapi: Rename .owner to .defined_in
QAPISchemaMember.owner is the name of the defining entity.  That's a
confusing name when an object type inherits members from a base type.
Rename it to .defined_in.  Rename .set_owner() and ._pretty_owner() to
match.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-3-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster
0ca7b11709 qapi: Tighten QAPISchemaFOO.check() assertions
When we introduced the QAPISchema intermediate representation (commit
ac88219a6c), we took a shortcut: we left check_exprs() & friends
alone instead of moving semantic checks into the
QAPISchemaFOO.check().  check_exprs() still checks and reports errors,
and the .check() assert check_exprs() did the job.  There are a few
gaps, though.

QAPISchemaArrayType.check() neglects to assert the element type is not
an array.  Add the assertion.

QAPISchemaObjectTypeVariants.check() neglects to assert the tag member
is not optional.  Add the assertion.

It neglects to assert the tag member is not conditional.  Add the
assertion.

It neglects to assert we actually have variants.  Add the assertion.

It asserts the variants are object types, but neglects to assert they
don't have variants.  Tighten the assertion.

QAPISchemaObjectTypeVariants.check_clash() has the same issue.
However, it can run only after .check().  Delete the assertion instead
of tightening it.

QAPISchemaAlternateType.check() neglects to assert the branch types
don't conflict.  Fixing that isn't trivial, so add just a TODO comment
for now.  It'll be resolved later in this series.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-2-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster
56176412e7 qapi: Assert .visit() and .check_clash() run only after .check()
Easy since the previous commit provides .checked.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-20-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
f9d1743b9b qapi: Fix excessive QAPISchemaEntity.check() recursion
Entity checking goes back to commit ac88219a6c "qapi: New QAPISchema
intermediate representation", v2.5.0.  It's designed to work as
follows: QAPISchema.check() calls .check() for all the schema's
entities.  An entity's .check() recurses into another entity's
.check() only if the C struct generated for the former contains the C
struct generated for the latter (pointers don't count).  This is used
to detect "object contains itself".

There are two instances of this:

* An object's C struct contains its base's C struct

  QAPISchemaObjectType.check() calls self.base.check()

* An object's C struct contains its variants' C structs

  QAPISchemaObjectTypeVariants().check calls v.type.check().  Since
  commit b807a1e1e3 "qapi: Check for QAPI collisions involving variant
  members", v2.6.0.

Thus, only object types can participate in recursion.

QAPISchemaObjectType.check() is made for that: it checks @self when
called the first time, recursing into base and variants, it reports an
"contains itself" error when this recursion reaches an object being
checked, and does nothing it reaches an object that has been checked
already.

The other .check() may safely assume they get called exactly once.

Sadly, this design has since eroded:

* QAPISchemaCommand.check() and QAPISchemaEvent.check() call
  .args_type.check().  Since commit c818408e44 "qapi: Implement boxed
  types for commands/events", v2.7.0.  Harmless, since args_type can
  only be an object type.

* QAPISchemaEntity.check() calls ._ifcond.check() when inheriting the
  condition from another type.  Since commit 4fca21c1b0 qapi: leave
  the ifcond attribute undefined until check(), v3.0.0.  This makes
  simple union wrapper types recurse into the wrapped type (nothing
  else uses this condition inheritance).  The .check() of types used
  as simple union branch type get called multiple times.

* QAPISchemaObjectType.check() calls its super type's .check()
  *before* the conditional handling multiple calls.  Also since commit
  4fca21c1b0.  QAPISchemaObjectType.check()'s guard against multiple
  checking doesn't protect QAPISchemaEntity.check().

* QAPISchemaArrayType.check() calls .element_type.check().  Also since
  commit 4fca21c1b0.  The .check() of types used as array element
  types get called multiple times.

  Commit 56a4689582 "qapi: Fix array first used in a different module"
  (v4.0.0) added more code relying on this .element_type.check().

The absence of explosions suggests the .check() involved happen to be
effectively idempotent.

Fix the unwanted recursion anyway:

* QAPISchemaCommand.check() and QAPISchemaEvent.check() calling
  .args_type.check() is unnecessary.  Delete the calls.

* Fix QAPISchemaObjectType.check() to call its super type's .check()
  after the conditional handling multiple calls.

* A QAPISchemaEntity's .ifcond becomes valid at .check().  This is due
  to arrays and simple unions.

  Most types get ifcond and info passed to their constructor.

  Array types don't: they get it from their element type, which
  becomes known only in .element_type.check().

  The implicit wrapper object types for simple union branches don't:
  they get it from the wrapped type, which might be an array.

  Ditch the idea to set .ifcond in .check().  Instead, turn it into a
  property and compute it on demand.  Safe because it's only used
  after the schema has been checked.

  Most types simply return the ifcond passed to their constructor.

  Array types forward to their .element_type instead, and the wrapper
  types forward to the wrapped type.

* A QAPISchemaEntity's .module becomes valid at .check().  This is
  because computing it needs info and schema.fname, and because array
  types get it from their element type instead.

  Make it a property just like .ifcond.

Additionally, have QAPISchemaEntity.check() assert it gets called at
most once, so the design invariant will stick this time.  Neglecting
that was entirely my fault.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-19-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Commit message tidied up]
2019-09-24 14:07:23 +02:00
Markus Armbruster
b1bc31f4b7 qapi: Fix to .check() empty structs just once
QAPISchemaObjectType.check() does nothing for types that have been
checked already.  Except the "has been checked" predicate is broken
for empty types: self.members is [] then, which isn't true.  The bug
is harmless, but fix it anyway: use self.member is not None instead.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-18-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
e31fe1266c qapi: Delete useless check_exprs() code for simple union kind
Commit bceae7697f "qapi script: support enum type as discriminator in
union" made check_exprs() add the implicit enum types of simple unions
to global @enum_types.  I'm not sure it was needed even then.  It's
certainly not needed now.  Delete it.

discriminator_find_enum_define() and add_name() parameter @implicit
are now dead.  Bury them.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-17-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
6955397677 qapi: Clean up around check_known_keys()
All callers pass a dict argument to @keys, except check_keys() passes
a dict's .keys().  Drop .keys() there, and rename parameter @keys to
@value.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-16-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
dc234189f8 qapi: Simplify check_keys()
check_keys() parameter expr_elem expects a dictionary with keys 'expr'
and 'info'.  Passing the two values separately is simpler, so do that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-15-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
fe9c4dcf90 qapi: Normalize 'if' in check_exprs(), like other sugar
We normalize shorthand to longhand forms in check_expr(): enumeration
values with normalize_enum(), feature values with
normalize_features(), struct members, union branches and alternate
branches with normalize_members().  If conditions are an exception: we
normalize them in QAPISchemaEntity.check() and
QAPISchemaMember.__init(), with listify_cond().  The idea goes back to
commit 2cbc94376e "qapi: pass 'if' condition into QAPISchemaEntity
objects", v3.0.0.

Normalize in check_expr() instead, with new helper normalize_if().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-14-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
dec0012ef8 qapi: Fix missing 'if' checks in struct, union, alternate 'data'
Commit 87adbbffd4..3e270dcacc "qapi: Add 'if' to (implicit
struct|union|alternate) members" (v4.0.0) neglected test coverage, and
promptly failed to check the conditions.  Review fail.

Recent commit "tests/qapi-schema: Demonstrate insufficient 'if'
checking" added test coverage, demonstrating the bug.  Fix it by add
the missing check_if().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-13-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
c2c7065e17 qapi: Reject blank 'if' conditions in addition to empty ones
"'if': 'COND'" generates "#if COND".  We reject empty COND because it
won't compile.  Blank COND won't compile any better, so reject that,
too.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-12-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
887a2069f7 qapi: Fix broken discriminator error messages
check_union() checks the discriminator exists in base and makes sense.
Two error messages mention the base.  These are broken for anonymous
bases, as demonstrated by tests flat-union-invalid-discriminator and
flat-union-invalid-if-discriminator.err.  The third one doesn't
bother.

First broken when commit ac4338f8eb "qapi: Allow anonymous base for
flat union" (v2.6.0) neglected to adjust the "not a member of base"
error message.  Commit ccadd6bcba "qapi: Add 'if' to implicit struct
members" (v4.0.0) then cloned the flawed error message.

Dumb them down not to mention the base.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-11-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
9d55380b5a qapi: Remove null from schema language
We represent the parse tree as OrderedDict.  We fetch optional dict
members with .get().  So far, so good.

We represent null literals as None.  .get() returns None both for
"absent" and for "present, value is the null literal".  Uh-oh.

Test features-if-invalid exposes this bug: "'if': null" is
misinterpreted as absent "if".

We added null to the schema language to "allow [...] an explicit
default value" (commit e53188ada5 "qapi: Allow true, false and null in
schema json", v2.4.0).  Hasn't happened; null is still unused except
as generic invalid value in tests/.

To fix, we'd have to replace .get() by something more careful, or
represent null differently.  Feasible, but we got more and bigger fish
to fry right now.  Remove the null literal from the schema language.
Replace null in tests by another invalid value.

Test features-if-invalid now behaves as it should.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-10-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
14c3279502 qapi: Improve reporting of lexical errors
Show text up to next structural character, whitespace, or quote
character instead of just the first character.

Forgotten quotes now get reported like "Stray 'command'" instead of
"Stray 'c'".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-9-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
9f5e6b088a qapi: Use quotes more consistently in frontend error messages
Consistently enclose error messages in double quotes.  Use single
quotes within, except for one case of "'".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-8-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
8d40738d2f qapi: Tweak code to match docs/devel/qapi-code-gen.txt
The previous commit made qapi-code-gen.txt define "(top-level)
expression" as either "directive" or "definition".  The code still
uses "expression" when it really means "definition".  Tidy up.

The previous commit made qapi-code-gen.txt use "object" rather than
"dictionary".  The code still uses "dictionary".  Tidy up.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-17-armbru@redhat.com>
2019-09-24 14:07:22 +02:00
Markus Armbruster
398969fe1c qapi: Adjust frontend errors to say enum value, not member
For consistency with docs/devel/qapi-code-gen.txt.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-12-armbru@redhat.com>
2019-09-24 14:07:22 +02:00
Markus Armbruster
0ced9531f1 qapi: Permit omitting all flat union branches
Absent flat union branches default to the empty struct (since commit
800877bb16 "qapi: allow empty branches in flat unions").  But an
attempt to omit all of them is rejected with "Union 'FOO' has no
branches".  Harmless oddity, but it's easy to avoid, so do that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-11-armbru@redhat.com>
[Commit message typo fixed]
2019-09-24 14:07:22 +02:00
Markus Armbruster
f03255362a qapi: Permit alternates with just one branch
A union or alternate without branches makes no sense and doesn't work:
it can't be instantiated.  A union or alternate with just one branch
works, but is degenerate.  We accept the former, but reject the
latter.  Weird.  docs/devel/qapi-code-gen.txt doesn't mention the
difference.  It claims an alternate definition is "is similar to a
simple union type".

Permit degenerate alternates to make them consistent with unions.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-10-armbru@redhat.com>
2019-09-24 14:07:22 +02:00
Markus Armbruster
675b214bc6 qapi: Permit 'boxed' with empty type
We reject empty types with 'boxed': true.  We don't really need that
to work, but making it work is actually simpler than rejecting it, so
do that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-9-armbru@redhat.com>
2019-09-24 14:07:22 +02:00
Markus Armbruster
9b4416bfc1 qapi: Drop support for escape sequences other than \\
Since the previous commit restricted strings to printable ASCII,
\uXXXX's only use is obfuscation.  Drop it.

This leaves \\, \/, \', and \".  Since QAPI schema strings are all
names, and names are restricted to ASCII letters, digits, hyphen, and
underscore, none of them is useful.

The latter three have no test coverage.  Drop them.

Keep \\ to avoid (more) gratuitous incompatibility with JSON.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-8-armbru@redhat.com>
2019-09-24 14:07:22 +02:00
Markus Armbruster
56a8caff92 qapi: Restrict strings to printable ASCII
RFC 8259 on string contents:

   All Unicode characters may be placed within the quotation marks,
   except for the characters that MUST be escaped: quotation mark,
   reverse solidus, and the control characters (U+0000 through
   U+001F).

The QAPI schema parser accepts both less and more than JSON: it
accepts only ASCII with \u (less), and accepts control characters
other than LF (new line) unescaped.  How it treats unescaped non-ASCII
input differs between Python 2 and Python 3.

Make it accept strictly less: require printable ASCII.  Drop support
for \b, \f, \n, \r, \t.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-7-armbru@redhat.com>
2019-09-24 14:07:22 +02:00
Markus Armbruster
b22e86585b qapi: Drop support for boxed alternate arguments
Commands and events can define their argument type inline (default) or
by referring to another type ('boxed': true, since commit c818408e44
"qapi: Implement boxed types for commands/events", v2.7.0).  The
unboxed inline definition is an (anonymous) struct type.  The boxed
type may be a struct, union, or alternate type.

The latter is problematic: docs/interop/qemu-spec.txt requires the
value of the 'data' key to be a json-object, but any non-degenerate
alternate type has at least one branch that isn't.

Fortunately, we haven't made use of alternates in this context outside
tests/.  Drop support for them.

QAPISchemaAlternateType.is_empty() is now unused.  Drop it, too.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-4-armbru@redhat.com>
2019-09-24 14:07:22 +02:00
Markus Armbruster
dcca907bed qapi: Drop check_type()'s redundant parameter @allow_optional
check_type() uses @allow_optional only when @value is a dictionary and
@allow_dict is True.  All callers that pass allow_dict=True also pass
allow_optional=True.

Therefore, @allow_optional is always True when check_type() uses it.
Drop the redundant parameter.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-3-armbru@redhat.com>
2019-09-24 14:07:22 +02:00
Markus Armbruster
b1862ee623 scripts/git.orderfile: Match QAPI schema more precisely
Pattern *.json also matches the tests/qapi-schema/*.json.  Separates
them from the tests/qapi-schema/*.{err,exit,out} in diffs.  I hate
that.  Change the pattern to match just the "real" QAPI schemata.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190913201349.24332-2-armbru@redhat.com>
2019-09-24 14:07:22 +02:00
Philippe Mathieu-Daudé
9f7ad79c16 trace: Forbid event format ending with newline character
Event format ending with newlines confuse the trace reports.
Forbid them.

Add a check to refuse new format added with trailing newline:

  $ make
  [...]
    GEN     hw/misc/trace.h
  Traceback (most recent call last):
    File "scripts/tracetool.py", line 152, in <module>
      main(sys.argv)
    File "scripts/tracetool.py", line 143, in main
      events.extend(tracetool.read_events(fh, arg))
    File "scripts/tracetool/__init__.py", line 367, in read_events
      event = Event.build(line)
    File "scripts/tracetool/__init__.py", line 281, in build
      raise ValueError("Event format can not end with a newline character")
  ValueError: Error at hw/misc/trace-events:121: Event format can not end with a newline character

Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20190916095121.29506-3-philmd@redhat.com
Message-Id: <20190916095121.29506-3-philmd@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2019-09-18 10:20:15 +01:00
Daniel P. Berrangé
336a7451e8 docs: convert README, CODING_STYLE and HACKING to RST syntax
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-09-05 14:27:06 +01:00
Marc-André Lureau
c5b35f03c3 minikconf: don't print CONFIG_FOO=n lines
qemu in general doesn't define CONFIG_FOO if it's false.  This also
helps with the dumb kconfig parser from meson, as source_set considers
any non-empty value as true.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-08-21 16:29:57 +02:00
Jan Kiszka
fc7d2b451e kvm: vmxcap: Enhance with latest features
Based on SDM from May 2019.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-08-20 20:00:51 +02:00
Richard Henderson
c692079597 decodetree: Suppress redundant declaration warnings
We can tell that a decodetree input file is "secondary" when it
uses an argument set marked "!extern".  This indicates that at
least one of the insn translation functions will have already
been declared by the "primary" input file, but given only the
secondary we cannot tell which.

Avoid redundant declaration warnings by suppressing them with pragmas.

Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Suggested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2019-08-19 08:13:14 -07:00
Richard Henderson
94597b6146 decodetree: Allow !function with no input bits
Call this form a "parameter", returning a value extracted
from the DisasContext.

Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2019-08-19 08:03:41 -07:00
Markus Armbruster
13d4ff07e8 trace: Do not include qom/cpu.h into generated trace.h
docs/devel/tracing.txt explains "since many source files include
trace.h, [the generated trace.h use] a minimum of types and other
header files included to keep the namespace clean and compile times
and dependencies down."

Commit 4815185902 "trace: Add per-vCPU tracing states for events with
the 'vcpu' property" made them all include qom/cpu.h via
control-internal.h.  qom/cpu.h in turn includes about thirty headers.
Ouch.

Per-vCPU tracing is currently not supported in sub-directories'
trace-events.  In other words, qom/cpu.h can only be used in
trace-root.h, not in any trace.h.

Split trace/control-vcpu.h off trace/control.h and
trace/control-internal.h.  Have the generated trace.h include
trace/control.h (which no longer includes qom/cpu.h), and trace-root.h
include trace/control-vcpu.h (which includes it).

The resulting improvement is a bit disappointing: in my "build
everything" tree, some 1100 out of 6600 objects (not counting tests
and objects that don't depend on qemu/osdep.h) depend on a trace.h,
and about 600 of them no longer depend on qom/cpu.h.  But more than
1300 others depend on trace-root.h.  More work is clearly needed.
Left for another day.

Cc: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190812052359.30071-8-armbru@redhat.com>
2019-08-16 13:31:52 +02:00
Markus Armbruster
988fa10304 trace: Eliminate use of TARGET_FMT_plx
hw/tpm/trace-events uses TARGET_FMT_plx formats with uint64_t
arguments.  That's wrong, TARGET_FMT_plx takes hwaddr.  Since hwaddr
happens to be uint64_t, it works anyway.  Messed up in commit
ec427498da, v2.12.0.  Clean up by replacing TARGET_FMT_plx with its
macro expansion.

scripts/tracetool/format/log_stap.py (commit 62dd1048c0, v4.0.0) has
a special case for TARGET_FMT_plx.  Delete it.

Cc: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20190812052359.30071-7-armbru@redhat.com>
2019-08-16 13:31:51 +02:00
Marc-André Lureau
84963b5bbf archive-source: also create a stash for submodules
"git archive" fails when a submodule has a modification, because "git
stash create" doesn't handle submodules. Let's teach our
archive-source.sh to handle modifications in submodules the same way
as qemu tree, by creating a stash.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20190708200250.12017-1-marcandre.lureau@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 15:53:25 +01:00
Paolo Bonzini
3cae16db02 create_config: remove $(CONFIG_SOFTMMU) hack
CONFIG_TPM is defined to a rather weird $(CONFIG_SOFTMMU) so that it
expands to the right thing in hw/Makefile.objs.  This however is not
needed anymore and it has a corresponding hack in create_config
to turn it into "#define CONFIG_TPM 1".  Clean up.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-07-15 20:58:37 +02:00
Paolo Bonzini
874acb6f83 checkpatch: detect doubly-encoded UTF-8
Copy and pasting from Thunderbird's "view source" window results in double
encoding of multibyte UTF-8 sequences.  The appearance of those sequences is
very peculiar, so detect it and give an error despite the (low) possibility
of false positives.

As the major offender, I am also adding the same check to my applypatch-msg
and commit-msg hooks, but this will also cause patchew to croak loudly when
this mistake happens.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <1558099140-53240-1-git-send-email-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-07-15 20:58:37 +02:00
Peter Maydell
c4107e8208 Bugfixes.
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQEcBAABAgAGBQJdH7FgAAoJEL/70l94x66DdUUH+gLr/ZdjLIdfYy9cjcnevf4E
 cJlxdaW9KvUsK2uVgqQ/3b1yF+GCGk10n6n8ZTIbClhs+6NpqEMz5O3FA/Na6FGA
 48M2DwJaJ2H9AG/lQBlSBNUZfLsEJ9rWy7DHvNut5XMJFuWGwdtF/jRhUm3KqaRq
 vAaOgcQHbzHU9W8r1NJ7l6pnPebeO7S0JQV+82T/ITTz2gEBDUkJ36boO6fedkVQ
 jLb9nZyG3CJXHm2WlxGO4hkqbLFzURnCi6imOh2rMdD8BCu1eIVl59tD1lC/A0xv
 Pp3xXnv9SgJXsV4/I/N3/nU85ZhGVMPQZXkxaajHPtJJ0rQq7FAG8PJMEj9yPe8=
 =poke
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

Bugfixes.

# gpg: Signature made Fri 05 Jul 2019 21:21:52 BST
# gpg:                using RSA key BFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream:
  ioapic: use irq number instead of vector in ioapic_eoi_broadcast
  hw/i386: Fix linker error when ISAPC is disabled
  Makefile: generate header file with the list of devices enabled
  target/i386: kvm: Fix when nested state is needed for migration
  minikconf: do not include variables from MINIKCONF_ARGS in config-all-devices.mak
  target/i386: fix feature check in hyperv-stub.c
  ioapic: clear irq_eoi when updating the ioapic redirect table entry
  intel_iommu: Fix unexpected unmaps during global unmap
  intel_iommu: Fix incorrect "end" for vtd_address_space_unmap
  i386/kvm: Fix build with -m32
  checkpatch: do not warn for multiline parenthesized returned value
  pc: fix possible NULL pointer dereference in pc_machine_get_device_memory_region_size()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-07-08 10:26:18 +01:00
Julio Montes
6c22ea9d83 Makefile: generate header file with the list of devices enabled
v2: generate config-devices.h which contains the list of devices enabled

Message-Id: <20190705143554.10295-1-julio.montes@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-07-05 22:19:58 +02:00
Paolo Bonzini
6b7ac49d57 minikconf: do not include variables from MINIKCONF_ARGS in config-all-devices.mak
When minikconf writes config-devices.mak, it includes all variables including
those from MINIKCONF_ARGS.  This causes values from config-host.mak to "stick" to
the ones used in generating config-devices.mak, because config-devices.mak is
included after config-host.mak.  Avoid this by omitting assignments coming
from the command line in the output of minikconf.

Reported-by: Christophe de Dinechin <dinechin@redhat.com>
Reviewed-by: Christophe de Dinechin <dinechin@redhat.com>
Tested-by: Christophe de Dinechin <dinechin@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-07-05 22:16:46 +02:00
Paolo Bonzini
c20b139620 checkpatch: do not warn for multiline parenthesized returned value
While indeed we do not want to have

    return (a);

it is less clear that this applies to

    return (a &&
            b);

Some editors indent more nicely if you have parentheses, and some people's
eyes may appreciate that as well.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <1561116534-21814-1-git-send-email-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-07-05 22:16:45 +02:00
John Snow
abf0bf998d python/qemu: split QEMUMachine out from underneath __init__.py
It's not obvious that something named __init__.py actually houses
important code that isn't relevant to python packaging glue. Move the
QEMUMachine and related error classes out into their own module.

Adjust users to the new import location.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190627212816.27298-2-jsnow@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-07-01 19:02:10 -03:00
Igor Mammedov
08aa87f5e7 qmp: make qmp-shell work with python3
python3 doesn't have raw_input(), so qmp-shell breaks.
Use input() instead and override it with raw_input()
if running on python2.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20190620154035.30989-1-imammedo@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-07-01 19:02:10 -03:00
Richard Henderson
2c7d442743 decodetree: Fix comparison of Field
Typo comparing the sign of the field, twice, instead of also comparing
the mask of the field (which itself encodes both position and length).

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190604154225.26992-1-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-06-13 15:14:03 +01:00
Peter Maydell
4747524f9f QAPI patches for 2019-06-12
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAl0BLAISHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZTlRMQAKLpFwk4kFlqYm0KHqBtdjn57Z756Boi
 9Uu/pDtet3+tpryMNcsM3lQ6uZCl35MY1ssYGCDc6JZVnLKgsN8EyO2MKnjAhogj
 hZWWhuJdR6vHGqgK+IyIOCgfIG26HS9+ejJIWJ4qm0sCAtmNEKjKRqJNPnSxbRHc
 wm1QnxFGikhKe74luDUk5zISlDFY1CILSk7W6K0nbdq6ByrMkq0dcjSLSpwWmALQ
 KSpWVkUz9CKxJ1dZn2iQ2af4VVKsL8xbl4R5nc5RInKFJ2W3fpVJqsXbEcZC6kiT
 WntZivbthbXXwc3I4IODs49Vu6PIt6OmH8vFmcWGt5hQB89ErhndVxtfCA11Qnml
 obhsQD+3AR4vncAJo+Y6yhbZQ0Fj2fy7xOrdPNXMdcXyLx9v7oOI66oXUpy4svT2
 TR0AK+8wipt8BihId6+YynC6I5hOMWDDjKJwCnWZ20rnnn/K+oLmG2BYpYpf0CmS
 3EE1SbdB/8eIhO0tc5Vyn3YegGkuAz0x0q3gGBTXvvk5ax0SffjXvgVLqb+aWcl4
 srfm5RwbWYvB+vynxCZvrPNikQWEqcOcIzBTvbpdkpsBpfX2pLBaYvHv91F960vR
 cjbo/CU+6Z4GMCHK43RXDAoZQmbjCBuqYer/+JL9Of2w7p5SnUIrBXgnKcg/SxEg
 VGxVLQmLYieP
 =YEV2
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2019-06-12' into staging

QAPI patches for 2019-06-12

# gpg: Signature made Wed 12 Jun 2019 17:44:50 BST
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2019-06-12:
  qapi: Simplify how QAPIDoc implements its state machine
  file-posix: Add dynamic-auto-read-only QAPI feature
  qapi: Allow documentation for features
  qapi: Disentangle QAPIDoc code
  tests/qapi-schema: Error case tests for features in structs
  tests/qapi-schema: Test for good feature lists in structs
  qapi: Add feature flags to struct types
  block/gluster: update .help of BLOCK_OPT_PREALLOC option
  block/file-posix: update .help of BLOCK_OPT_PREALLOC option
  qapi/block-core: update documentation of preallocation parameter
  qdev: Delete unused LostTickPolicy "merge"

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-06-13 11:58:00 +01:00