15 Commits
Author SHA1 Message Date
Wez Furlong 051b85ba7d break out various test configurations as their own TestCase instances
This allows the tests to be distributed across the test runner threads
more effectively.

Also fix up a line buffering issue with the Json codec
2015-08-02 15:10:57 -07:00
Wez Furlong 240c8c9ebf allow running tests in parallel in runtests.py 2015-08-02 14:39:15 -07:00
Wez Furlong 0b18152104 pywatchman: fixup bser handling for python long values
Also wire up py-build into `make integration`
2015-08-01 00:31:29 -07:00
Wez Furlong 89bcf21507 pywatchman: support subscriptions and log responses 2015-08-01 00:27:26 -07:00
Wez Furlong 75c5909f61 watchman: allow running various test configurations in python tests
Tests combinations of bser, json, local IPC and CLI interfaces
2015-07-28 21:15:52 -07:00
Wez Furlong e7da42b204 watchman: start building python based test suite
Summary:
We've had a number of compatibility issues with the arcanist
based test suite, so this is an effort to run them using the
python unittest infrastructure.

This new structure allows us to create a single temporary dir
and to create a dir per test case to track the temporary files
and dirs created during the test.  This is beneficial both from
a post-mortem perspective if a test fails, but also because
the paths that show up in the watchman logs will now be easily
recognizable as being associated with a given test.

This will also help us manage the windows integration tests
(https://github.com/facebook/watchman/issues/19#issuecomment-125054948)
a bit more sanely; a source of errors in the php tests is that
deleting directory trees can fail if a handle still references
any part of it, and there is often a noticeable lag where we
can hit this state and error out.  By deferring the deletes
until our process end, we should minimize this issue.

I've ported a single integration test to demonstrate what this
looks like, the rest will have to be a series of diffs for
easier review.

Test Plan: `make integration` or `./runtests.py`

Reviewers: sid0

Differential Revision: https://reviews.facebook.net/D43137
2015-07-28 21:15:47 -07:00
Siddharth Agarwal 3f9e9e9f7d pywatchman: add a docstring 2015-06-29 22:33:04 -07:00
Siddharth Agarwal 0ba4bbb42c pywatchman: rename Unavailable to WatchmanError and add CommandError 2015-06-29 22:30:28 -07:00
Marc Horowitz b5bc4abcd5 refactor receive() code out of query() so it can be used with notifications
Summary:
this is a change to upstream third-party source, and should
be submitted for inclusion.

Test Plan: I have an application which uses this, and it works.

Reviewers: sid0, wez

Reviewed By: wez

Differential Revision: https://reviews.facebook.net/D28641
2014-11-11 17:38:00 -08:00
Wez Furlong e3c9d74ee1 read minimum length when determining pdu length
in cases where the session is long lived (eg: subscriptions), reading
too much without buffering could lead to skipping over multiple small
responses.
2014-05-17 18:17:16 -07:00
Wez Furlong 6ea4ed8343 refactor python bser module to pywatchman.bser, add pywatchman.client
Summary:
This draws from our hgwatchman client code to provide a simple
watchman client API for python code.

This changes the makefile so that we install our watchman python module
during `make install`.  It also wires up a rudimentary test of the
client that talks to one of the instances set up by
WatchmanIntegrationEngine.

Test Plan: `./configure --with-python ; make integration`

Reviewers: sid0, achao

Reviewed By: sid0

Subscribers: moshez

Differential Revision: https://phabricator.fb.com/D1323720
2014-05-16 13:11:03 -07:00
Wez Furlong 773accf4df avoid misaligned pointer access in python bser module
Addresses https://github.com/facebook/watchman/issues/26

I don't have a sparc handy to test this on, but this passes our tests,
so I'm calling this good enough.
2014-04-20 10:23:10 -07:00
Wez Furlong 8a2baf07e3 An exception to our Apache License
The file python/bser/bser.c is made available under a 3-clause BSD
license to facilitate linkage with some other projects.
2013-05-23 14:18:24 -07:00
Wez Furlong ec00509ebf clean up bser module for python
Integrate python build more fully with the overal build/test code.
Fixup some integer size issues and raise errors when integers are
too large for python to handle.

Allow arc unit to exercise python unit test.
2013-05-21 00:03:36 -07:00
Wez Furlong 6550b9f6eb BSER: a binary serialization format
Summary:
Add a binary serialization format.  This inserts fairly cleanly into our handling:

 * Format can represent all JSON types implemented in our library
 * Strings are no longer spec'd as being UTF-8; they're binary
 * The CLI defaults to bser for client<->server communication and JSON for output
 * CLI options to select output and server encoding formats
 * The server responds in the same format as the request it received

The binary format describes its own length which allows clients to know ahead of
time how much memory they'll need and avoid realloc.  Each data item is a copy
of its binary representation on the server, except for integer types which are
encoded using the smallest possible representation for the value at hand.  This
allows for a degree of compression compared to textual JSON and is much simpler
to decode than translating string representations of numbers back and forth.

Integers are presented in host byte order since communication is always local;
it makes no sense to byteswap from LE to BE and back again.

Further compression is realized by changing the JSON library API to
define "template" objects with properties that are known at the start of
array processing; this allows the property names to be factored out of
the object value stream and allows saving approximately 50% of the space
vs. compact JSON.

A python `bser` extension is included that supports `dumps` and `loads`;
you can exercise the extension using:

```
cd python/bser
./setup.py build
PYTHONPATH=build/lib.linux-x86_64-2.6 ./tests.py
```

Reviewers: sid0

Reviewed By: sid0

CC: security-diffs@lists, bryano

Differential Revision: https://phabricator.fb.com/D809287

Task ID: 2339174
2013-05-20 17:47:23 -07:00