Magic-Mirror/gecko-dev
1
0
Fork 0
mirror of https://github.com/mozilla/gecko-dev.git synced 2024-10-12 21:05:36 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge mozilla-central into mozilla-inbound

Browse Source
This commit is contained in:
Ehsan Akhgari 2012-06-04 11:32:30 -04:00
commit 0cf0edd3a9
86 changed files with 6727 additions and 1637 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
browser/installer/package-manifest.in
View File

@ -442,7 +442,9 @@
@BINPATH@/components/nsPrompter.js
#ifdef MOZ_SERVICES_SYNC
@BINPATH@/components/SyncComponents.manifest
@BINPATH@/components/AitcComponents.manifest
@BINPATH@/components/Weave.js
@BINPATH@/components/Aitc.js
#endif
@BINPATH@/components/TelemetryPing.js
@BINPATH@/components/TelemetryPing.manifest
@ -517,6 +519,7 @@
@BINPATH@/@PREF_DIR@/firefox-branding.js
#ifdef MOZ_SERVICES_SYNC
@BINPATH@/@PREF_DIR@/services-sync.js
@BINPATH@/@PREF_DIR@/services-aitc.js
#endif
@BINPATH@/greprefs.js
@BINPATH@/defaults/autoconfig/platform.js

426
build/pylib/blessings/blessings.egg-info/PKG-INFO
View File

@ -1,426 +0,0 @@
Metadata-Version: 1.0
Name: blessings
Version: 1.3
Summary: A thin, practical wrapper around terminal formatting, positioning, and more
Home-page: https://github.com/erikrose/blessings
Author: Erik Rose
Author-email: erikrose@grinchcentral.com
License: MIT
Description: =========
Blessings
=========
Coding with Blessings looks like this... ::
from blessings import Terminal
t = Terminal()
print t.bold('Hi there!')
print t.bold_red_on_bright_green('It hurts my eyes!')
with t.location(0, t.height - 1):
print 'This is at the bottom.'
Or, for byte-level control, you can drop down and play with raw terminal
capabilities::
print '{t.bold}All your {t.red}bold and red base{t.normal}'.format(t=t)
print t.wingo(2)
The Pitch
=========
Blessings lifts several of curses_' limiting assumptions, and it makes your
code pretty, too:
* Use styles, color, and maybe a little positioning without clearing the whole
screen first.
* Leave more than one screenful of scrollback in the buffer after your program
exits, like a well-behaved command-line app should.
* Get rid of all those noisy, C-like calls to ``tigetstr`` and ``tparm``, so
your code doesn't get crowded out by terminal bookkeeping.
* Act intelligently when somebody redirects your output to a file, omitting the
terminal control codes the user doesn't want to see (optional).
.. _curses: http://docs.python.org/library/curses.html
Before And After
----------------
Without Blessings, this is how you'd print some underlined text at the bottom
of the screen::
from curses import tigetstr, setupterm, tparm
from fcntl import ioctl
from os import isatty
import struct
import sys
from termios import TIOCGWINSZ
# If we want to tolerate having our output piped to other commands or
# files without crashing, we need to do all this branching:
if hasattr(sys.stdout, 'fileno') and isatty(sys.stdout.fileno()):
setupterm()
sc = tigetstr('sc')
cup = tigetstr('cup')
rc = tigetstr('rc')
underline = tigetstr('smul')
normal = tigetstr('sgr0')
else:
sc = cup = rc = underline = normal = ''
print sc # Save cursor position.
if cup:
# tigetnum('lines') doesn't always update promptly, hence this:
height = struct.unpack('hhhh', ioctl(0, TIOCGWINSZ, '\000' * 8))[0]
print tparm(cup, height - 1, 0) # Move cursor to bottom.
print 'This is {under}underlined{normal}!'.format(under=underline,
normal=normal)
print rc # Restore cursor position.
Phew! That was long and full of incomprehensible trash! Let's try it again,
this time with Blessings::
from blessings import Terminal
term = Terminal()
with term.location(0, term.height - 1):
print 'This is', term.underline('pretty!')
Much better.
What It Provides
================
Blessings provides just one top-level object: ``Terminal``. Instantiating a
``Terminal`` figures out whether you're on a terminal at all and, if so, does
any necessary terminal setup. After that, you can proceed to ask it all sorts
of things about the terminal. Terminal terminal terminal.
Simple Formatting
-----------------
Lots of handy formatting codes ("capabilities" in low-level parlance) are
available as attributes on a ``Terminal``. For example::
from blessings import Terminal
term = Terminal()
print 'I am ' + term.bold + 'bold' + term.normal + '!'
You can also use them as wrappers so you don't have to say ``normal``
afterward::
print 'I am', term.bold('bold') + '!'
Or, if you want fine-grained control while maintaining some semblance of
brevity, you can combine it with Python's string formatting, which makes
attributes easy to access::
print 'All your {t.red}base {t.underline}are belong to us{t.normal}'.format(t=term)
Simple capabilities of interest include...
* ``bold``
* ``reverse``
* ``underline``
* ``no_underline`` (which turns off underlining)
* ``blink``
* ``normal`` (which turns off everything, even colors)
* ``clear_eol`` (clear to the end of the line)
* ``clear_bol`` (clear to beginning of line)
* ``clear_eos`` (clear to end of screen)
Here are a few more which are less likely to work on all terminals:
* ``dim``
* ``italic`` and ``no_italic``
* ``shadow`` and ``no_shadow``
* ``standout`` and ``no_standout``
* ``subscript`` and ``no_subscript``
* ``superscript`` and ``no_superscript``
* ``flash`` (which flashes the screen once)
Note that, while the inverse of ``underline`` is ``no_underline``, the only way
to turn off ``bold`` or ``reverse`` is ``normal``, which also cancels any
custom colors. This is because there's no way to tell the terminal to undo
certain pieces of formatting, even at the lowest level.
You might notice that the above aren't the typical incomprehensible terminfo
capability names; we alias a few of the harder-to-remember ones for
readability. However, you aren't limited to these: you can reference any
string-returning capability listed on the `terminfo man page`_ by the name
under the "Cap-name" column: for example, ``term.rum``.
.. _`terminfo man page`: http://www.manpagez.com/man/5/terminfo/
Color
-----
16 colors, both foreground and background, are available as easy-to-remember
attributes::
from blessings import Terminal
term = Terminal()
print term.red + term.on_green + 'Red on green? Ick!' + term.normal
print term.bright_red + term.on_bright_blue + 'This is even worse!' + term.normal
You can also call them as wrappers, which sets everything back to normal at the
end::
print term.red_on_green('Red on green? Ick!')
print term.yellow('I can barely see it.')
The available colors are...
* ``black``
* ``red``
* ``green``
* ``yellow``
* ``blue``
* ``magenta``
* ``cyan``
* ``white``
You can set the background color instead of the foreground by prepending
``on_``, as in ``on_blue``. There is also a ``bright`` version of each color:
for example, ``on_bright_blue``.
There is also a numerical interface to colors, which takes an integer from
0-15::
term.color(5) + 'Hello' + term.normal
term.on_color(3) + 'Hello' + term.normal
term.color(5)('Hello')
term.on_color(3)('Hello')
If some color is unsupported (for instance, if only the normal colors are
available, not the bright ones), trying to use it will, on most terminals, have
no effect: the foreground and background colors will stay as they were. You can
get fancy and do different things depending on the supported colors by checking
`number_of_colors`_.
.. _`number_of_colors`: http://packages.python.org/blessings/#blessings.Terminal.number_of_colors
Compound Formatting
-------------------
If you want to do lots of crazy formatting all at once, you can just mash it
all together::
from blessings import Terminal
term = Terminal()
print term.bold_underline_green_on_yellow + 'Woo' + term.normal
Or you can use your newly coined attribute as a wrapper, which implicitly sets
everything back to normal afterward::
print term.bold_underline_green_on_yellow('Woo')
This compound notation comes in handy if you want to allow users to customize
the formatting of your app: just have them pass in a format specifier like
"bold_green" on the command line, and do a quick ``getattr(term,
that_option)('Your text')`` when you do your formatting.
I'd be remiss if I didn't credit couleur_, where I probably got the idea for
all this mashing.
.. _couleur: http://pypi.python.org/pypi/couleur
Parametrized Capabilities
-------------------------
Some capabilities take parameters. Rather than making you dig up ``tparm()``
all the time, we simply make such capabilities into callable strings. You can
pass the parameters right in::
from blessings import Terminal
term = Terminal()
print term.move(10, 1)
Here are some of interest:
``move``
Position the cursor elsewhere. Parameters are y coordinate, then x
coordinate.
``move_x``
Move the cursor to the given column.
``move_y``
Move the cursor to the given row.
You can also reference any other string-returning capability listed on the
`terminfo man page`_ by its name under the "Cap-name" column.
.. _`terminfo man page`: http://www.manpagez.com/man/5/terminfo/
Height and Width
----------------
It's simple to get the height and width of the terminal, in characters::
from blessings import Terminal
term = Terminal()
height = term.height
width = term.width
These are newly updated each time you ask for them, so they're safe to use from
SIGWINCH handlers.
Temporary Repositioning
-----------------------
Sometimes you need to flit to a certain location, print something, and then
return: for example, when updating a progress bar at the bottom of the screen.
``Terminal`` provides a context manager for doing this concisely::
from blessings import Terminal
term = Terminal()
with term.location(0, term.height - 1):
print 'Here is the bottom.'
print 'This is back where I came from.'
Parameters to ``location()`` are ``x`` and then ``y``, but you can also pass
just one of them, leaving the other alone. For example... ::
with term.location(y=10):
print 'We changed just the row.'
If you want to reposition permanently, see ``move``, in an example above.
Pipe Savvy
----------
If your program isn't attached to a terminal, like if it's being piped to
another command or redirected to a file, all the capability attributes on
``Terminal`` will return empty strings. You'll get a nice-looking file without
any formatting codes gumming up the works.
If you want to override this--like if you anticipate your program being piped
through ``less -r``, which handles terminal escapes just fine--pass
``force_styling=True`` to the ``Terminal`` constructor.
In any case, there is an ``is_a_tty`` attribute on ``Terminal`` that lets you
see whether the attached stream seems to be a terminal. If it's false, you
might refrain from drawing progress bars and other frippery, since you're
apparently headed into a pipe::
from blessings import Terminal
term = Terminal()
if term.is_a_tty:
with term.location(0, term.height - 1):
print 'Progress: [=======> ]'
print term.bold('Important stuff')
Shopping List
=============
There are decades of legacy tied up in terminal interaction, so attention to
detail and behavior in edge cases make a difference. Here are some ways
Blessings has your back:
* Uses the terminfo database so it works with any terminal type
* Provides up-to-the-moment terminal height and width, so you can respond to
terminal size changes (SIGWINCH signals). (Most other libraries query the
``COLUMNS`` and ``LINES`` environment variables or the ``cols`` or ``lines``
terminal capabilities, which don't update promptly, if at all.)
* Avoids making a mess if the output gets piped to a non-terminal
* Works great with standard Python string templating
* Provides convenient access to all terminal capabilities, not just a sugared
few
* Outputs to any file-like object, not just stdout
* Keeps a minimum of internal state, so you can feel free to mix and match with
calls to curses or whatever other terminal libraries you like
Blessings does not provide...
* Native color support on the Windows command prompt. However, it should work
when used in concert with colorama_.
.. _colorama: http://pypi.python.org/pypi/colorama/0.2.4
Bugs
====
Bugs or suggestions? Visit the `issue tracker`_.
.. _`issue tracker`: https://github.com/erikrose/blessings/issues/new
License
=======
Blessings is under the MIT License. See the LICENSE file.
Version History
===============
1.3
* Add ``number_of_colors``, which tells you how many colors the terminal
supports.
* Made ``color(n)`` and ``on_color(n)`` callable to wrap a string, like the
named colors can. Also, make them both fall back to the ``setf`` and
``setb`` capabilities (like the named colors do) if the ANSI ``setaf`` and
``setab`` aren't available.
* Allow ``color`` attr to act as an unparametrized string, not just a
callable.
* Make ``height`` and ``width`` examine any passed-in stream before falling
back to stdout. (This rarely if ever affects actual behavior; it's mostly
philosophical.)
* Make caching simpler and slightly more efficient.
* Get rid of a reference cycle between Terminals and FormattingStrings.
* Update docs to reflect that terminal addressing (as in ``location()``) is
0-based.
1.2
* Added support for Python 3! We need 3.2.3 or greater, because the curses
library couldn't decide whether to accept strs or bytes before that
(http://bugs.python.org/issue10570).
* Everything that comes out of the library is now unicode. This lets us
support Python 3 without making a mess of the code, and Python 2 should
continue to work unless you were testing types (and badly). Please file a
bug if this causes trouble for you.
* Changed to the MIT License for better world domination.
* Added Sphinx docs.
1.1
* Added nicely named attributes for colors.
* Introduced compound formatting.
* Added wrapper behavior for styling and colors.
* Let you force capabilities to be non-empty, even if the output stream is
not a terminal.
* Added the ``is_a_tty`` attribute for telling whether the output stream is a
terminal.
* Sugared the remaining interesting string capabilities.
* Let ``location()`` operate on just an x *or* y coordinate.
1.0
* Extracted Blessings from nose-progressive, my `progress-bar-having,
traceback-shortcutting, rootin', tootin' testrunner`_. It provided the
tootin' functionality.
.. _`progress-bar-having, traceback-shortcutting, rootin', tootin' testrunner`: http://pypi.python.org/pypi/nose-progressive/
Keywords: terminal,tty,curses,ncurses,formatting,style,color,console
Platform: UNKNOWN
Classifier: Intended Audience :: Developers
Classifier: Natural Language :: English
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Environment :: Console :: Curses
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.5
Classifier: Programming Language :: Python :: 2.6
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.2
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: User Interfaces
Classifier: Topic :: Terminals

12
build/pylib/blessings/blessings.egg-info/SOURCES.txt
View File

@ -1,12 +0,0 @@
LICENSE
MANIFEST.in
README.rst
setup.cfg
setup.py
tox.ini
blessings/__init__.py
blessings/tests.py
blessings.egg-info/PKG-INFO
blessings.egg-info/SOURCES.txt
blessings.egg-info/dependency_links.txt
blessings.egg-info/top_level.txt

1
build/pylib/blessings/blessings.egg-info/dependency_links.txt
View File

@ -1 +0,0 @@

1
build/pylib/blessings/blessings.egg-info/top_level.txt
View File

@ -1 +0,0 @@
blessings

4
dom/file/FileHandle.cpp
View File

@ -72,8 +72,8 @@ NS_INTERFACE_MAP_END_INHERITING(nsDOMEventTargetHelper)
NS_IMPL_ADDREF_INHERITED(FileHandle, nsDOMEventTargetHelper)
NS_IMPL_RELEASE_INHERITED(FileHandle, nsDOMEventTargetHelper)
NS_IMPL_EVENT_HANDLER(FileHandle, abort);
NS_IMPL_EVENT_HANDLER(FileHandle, error);
NS_IMPL_EVENT_HANDLER(FileHandle, abort)
NS_IMPL_EVENT_HANDLER(FileHandle, error)
NS_IMETHODIMP
FileHandle::GetName(nsAString& aName)

2
dom/file/FileRequest.cpp
View File

@ -135,7 +135,7 @@ NS_IMPL_RELEASE_INHERITED(FileRequest, DOMRequest)
DOMCI_DATA(FileRequest, FileRequest)
NS_IMPL_EVENT_HANDLER(FileRequest, progress);
NS_IMPL_EVENT_HANDLER(FileRequest, progress)
void
FileRequest::FireProgressEvent(PRUint64 aLoaded, PRUint64 aTotal)

18
dom/file/LockedFile.cpp
View File

@ -349,9 +349,9 @@ NS_IMPL_RELEASE_INHERITED(LockedFile, nsDOMEventTargetHelper)
DOMCI_DATA(LockedFile, LockedFile)
NS_IMPL_EVENT_HANDLER(LockedFile, complete);
NS_IMPL_EVENT_HANDLER(LockedFile, abort);
NS_IMPL_EVENT_HANDLER(LockedFile, error);
NS_IMPL_EVENT_HANDLER(LockedFile, complete)
NS_IMPL_EVENT_HANDLER(LockedFile, abort)
NS_IMPL_EVENT_HANDLER(LockedFile, error)
nsresult
LockedFile::PreHandleEvent(nsEventChainPreVisitor& aVisitor)
@ -662,7 +662,9 @@ LockedFile::Append(const jsval& aValue,
}
NS_IMETHODIMP
LockedFile::Truncate(nsIDOMFileRequest** _retval)
LockedFile::Truncate(PRUint64 aLocation,
PRUint8 aOptionalArgCount,
nsIDOMFileRequest** _retval)
{
NS_ASSERTION(NS_IsMainThread(), "Wrong thread!");
@ -682,12 +684,18 @@ LockedFile::Truncate(nsIDOMFileRequest** _retval)
nsRefPtr<FileRequest> fileRequest = GenerateFileRequest();
NS_ENSURE_TRUE(fileRequest, NS_ERROR_DOM_FILEHANDLE_UNKNOWN_ERR);
PRUint64 location = aOptionalArgCount ? aLocation : mLocation;
nsRefPtr<TruncateHelper> helper =
new TruncateHelper(this, fileRequest, mLocation);
new TruncateHelper(this, fileRequest, location);
nsresult rv = helper->Enqueue();
NS_ENSURE_SUCCESS(rv, NS_ERROR_DOM_FILEHANDLE_UNKNOWN_ERR);
if (aOptionalArgCount) {
mLocation = aLocation;
}
fileRequest.forget(_retval);
return NS_OK;
}

5
dom/file/nsIDOMLockedFile.idl
View File

@ -16,7 +16,7 @@ dictionary DOMFileMetadataParameters
boolean lastModified;
};
[scriptable, builtinclass, uuid(63055eeb-cc19-468b-bafa-7b7961796340)]
[scriptable, builtinclass, uuid(e1f69cc5-c6ce-4850-bc09-c4211b1d4290)]
interface nsIDOMLockedFile : nsISupports
{
readonly attribute nsIDOMFileHandle fileHandle;
@ -49,8 +49,9 @@ interface nsIDOMLockedFile : nsISupports
nsIDOMFileRequest
append(in jsval value);
[optional_argc]
nsIDOMFileRequest
truncate();
truncate([optional] in unsigned long long location);
nsIDOMFileRequest
flush();

23
dom/file/test/test_truncate.html
View File

@ -31,7 +31,7 @@
is(lockedFile.location, 100000, "Correct location");
for (let i = 0; i < 10; i++) {
let location = lockedFile.location - 10000
let location = lockedFile.location - 10000;
lockedFile.location = location;
request = lockedFile.truncate();
@ -46,6 +46,27 @@
is(event.target.result.size, location, "Correct size");
}
request = lockedFile.write(testBuffer);
request.onsuccess = grabEventAndContinueHandler;
event = yield;
let location = lockedFile.location;
for (let i = 0; i < 10; i++) {
location -= 10000;
request = lockedFile.truncate(location);
request.onsuccess = grabEventAndContinueHandler;
event = yield;
is(lockedFile.location, location, "Correct location");
request = lockedFile.getMetadata({ size: true });
request.onsuccess = grabEventAndContinueHandler;
event = yield;
is(event.target.result.size, location, "Correct size");
}
}
finishTest();

8
dom/indexedDB/ipc/IndexedDBParent.cpp
View File

@ -239,7 +239,7 @@ IndexedDBDatabaseParent::HandleRequestEvent(nsIDOMEvent* aEvent,
nsCOMPtr<nsIIDBVersionChangeEvent> changeEvent = do_QueryInterface(aEvent);
NS_ENSURE_TRUE(changeEvent, NS_ERROR_FAILURE);
uint64_t oldVersion;
PRUint64 oldVersion;
rv = changeEvent->GetOldVersion(&oldVersion);
NS_ENSURE_SUCCESS(rv, rv);
@ -333,7 +333,7 @@ IndexedDBDatabaseParent::HandleRequestEvent(nsIDOMEvent* aEvent,
nsCOMPtr<nsIIDBVersionChangeEvent> changeEvent = do_QueryInterface(aEvent);
NS_ENSURE_TRUE(changeEvent, NS_ERROR_FAILURE);
uint64_t oldVersion;
PRUint64 oldVersion;
rv = changeEvent->GetOldVersion(&oldVersion);
NS_ENSURE_SUCCESS(rv, rv);
@ -383,7 +383,7 @@ IndexedDBDatabaseParent::HandleDatabaseEvent(nsIDOMEvent* aEvent,
nsCOMPtr<nsIIDBVersionChangeEvent> changeEvent = do_QueryInterface(aEvent);
NS_ENSURE_TRUE(changeEvent, NS_ERROR_FAILURE);
uint64_t oldVersion;
PRUint64 oldVersion;
rv = changeEvent->GetOldVersion(&oldVersion);
NS_ENSURE_SUCCESS(rv, rv);
@ -1757,7 +1757,7 @@ IndexedDBDeleteDatabaseRequestParent::HandleEvent(nsIDOMEvent* aEvent)
nsCOMPtr<nsIIDBVersionChangeEvent> event = do_QueryInterface(aEvent);
MOZ_ASSERT(event);
uint64_t currentVersion;
PRUint64 currentVersion;
rv = event->GetOldVersion(&currentVersion);
NS_ENSURE_SUCCESS(rv, rv);

37
dom/tests/mochitest/ajax/offline/744719-cancel.cacheManifest Normal file
View File

@ -0,0 +1,37 @@
CACHE MANIFEST
http://mochi.test:8888/tests/SimpleTest/SimpleTest.js
http://mochi.test:8888/tests/dom/tests/mochitest/ajax/offline/offlineTests.js
# more than 15 what is a number of parallel loads
subresource744719.html?001
subresource744719.html?002
subresource744719.html?003
subresource744719.html?004
subresource744719.html?005
subresource744719.html?006
subresource744719.html?007
subresource744719.html?008
subresource744719.html?009
# this one is non existing and should cancel the load
nonexistent744719.html?010
subresource744719.html?011
subresource744719.html?012
subresource744719.html?013
subresource744719.html?014
subresource744719.html?015
subresource744719.html?016
subresource744719.html?017
subresource744719.html?018
subresource744719.html?019
subresource744719.html?020
subresource744719.html?021
subresource744719.html?022
subresource744719.html?023
subresource744719.html?024
subresource744719.html?025
subresource744719.html?026
subresource744719.html?027
subresource744719.html?028
subresource744719.html?029
subresource744719.html?030

2
dom/tests/mochitest/ajax/offline/744719-cancel.cacheManifest^headers^ Normal file
View File

@ -0,0 +1,2 @@
Content-Type: text/cache-manifest

36
dom/tests/mochitest/ajax/offline/744719.cacheManifest Normal file
View File

@ -0,0 +1,36 @@
CACHE MANIFEST
http://mochi.test:8888/tests/SimpleTest/SimpleTest.js
http://mochi.test:8888/tests/dom/tests/mochitest/ajax/offline/offlineTests.js
# more than 15 what is a number of parallel loads
subresource744719.html?001
subresource744719.html?002
subresource744719.html?003
subresource744719.html?004
subresource744719.html?005
subresource744719.html?006
subresource744719.html?007
subresource744719.html?008
subresource744719.html?009
subresource744719.html?010
subresource744719.html?011
subresource744719.html?012
subresource744719.html?013
subresource744719.html?014
subresource744719.html?015
subresource744719.html?016
subresource744719.html?017
subresource744719.html?018
subresource744719.html?019
subresource744719.html?020
subresource744719.html?021
subresource744719.html?022
subresource744719.html?023
subresource744719.html?024
subresource744719.html?025
subresource744719.html?026
subresource744719.html?027
subresource744719.html?028
subresource744719.html?029
subresource744719.html?030

2
dom/tests/mochitest/ajax/offline/744719.cacheManifest^headers^ Normal file
View File

@ -0,0 +1,2 @@
Content-Type: text/cache-manifest

7
dom/tests/mochitest/ajax/offline/Makefile.in
View File

@ -31,6 +31,13 @@ _TEST_FILES = \
test_bug460353.html \
test_bug474696.html \
test_bug544462.html \
test_bug744719.html \
744719.cacheManifest \
744719.cacheManifest^headers^ \
test_bug744719-cancel.html \
744719-cancel.cacheManifest \
744719-cancel.cacheManifest^headers^ \
subresource744719.html \
test_foreign.html \
test_fallback.html \
test_overlap.html \

1
dom/tests/mochitest/ajax/offline/subresource744719.html Normal file
View File

@ -0,0 +1 @@
<html><body>Dummy subresource</body></html>

83
dom/tests/mochitest/ajax/offline/test_bug744719-cancel.html Normal file
View File

@ -0,0 +1,83 @@
<html xmlns="http://www.w3.org/1999/xhtml" manifest="http://mochi.test:8888/tests/dom/tests/mochitest/ajax/offline/744719-cancel.cacheManifest">
<head>
<title>parallel load canceled</title>
<script type="text/javascript" src="/tests/SimpleTest/SimpleTest.js"></script>
<script type="text/javascript" src="/tests/dom/tests/mochitest/ajax/offline/offlineTests.js"></script>
<link rel="stylesheet" type="text/css" href="/tests/SimpleTest/test.css" />
<script type="text/javascript">
/*
Manifest refers a large number of resource to load. The 10th item however is a reference to a non-existing
resource that cancels the load. This test checks we cancel all loads and don't leak any of the other resources
after cancelation.
*/
ok(applicationCache.mozItems.length == 0,
"applicationCache.mozItems should be available and empty before associating with a cache.");
function updateCanceled()
{
OfflineTest.checkCache("http://mochi.test:8888/tests/dom/tests/mochitest/ajax/offline/744719-cancel.cacheManifest", false);
OfflineTest.checkCache("http://mochi.test:8888/tests/SimpleTest/SimpleTest.js", false);
OfflineTest.checkCache("http://mochi.test:8888/tests/dom/tests/mochitest/ajax/offline/offlineTests.js", false);
OfflineTest.checkCache("http://mochi.test:8888/tests/dom/tests/mochitest/ajax/offline/nonexistent744719?010", false);
var URL = "http://mochi.test:8888/tests/dom/tests/mochitest/ajax/offline/subresource744719.html?";
OfflineTest.checkCache(URL + "001", false);
OfflineTest.checkCache(URL + "002", false);
OfflineTest.checkCache(URL + "003", false);
OfflineTest.checkCache(URL + "004", false);
OfflineTest.checkCache(URL + "005", false);
OfflineTest.checkCache(URL + "006", false);
OfflineTest.checkCache(URL + "007", false);
OfflineTest.checkCache(URL + "008", false);
OfflineTest.checkCache(URL + "009", false);
OfflineTest.checkCache(URL + "011", false);
OfflineTest.checkCache(URL + "012", false);
OfflineTest.checkCache(URL + "013", false);
OfflineTest.checkCache(URL + "014", false);
OfflineTest.checkCache(URL + "015", false);
OfflineTest.checkCache(URL + "016", false);
OfflineTest.checkCache(URL + "017", false);
OfflineTest.checkCache(URL + "018", false);
OfflineTest.checkCache(URL + "019", false);
OfflineTest.checkCache(URL + "020", false);
OfflineTest.checkCache(URL + "021", false);
OfflineTest.checkCache(URL + "022", false);
OfflineTest.checkCache(URL + "023", false);
OfflineTest.checkCache(URL + "024", false);
OfflineTest.checkCache(URL + "025", false);
OfflineTest.checkCache(URL + "026", false);
OfflineTest.checkCache(URL + "027", false);
OfflineTest.checkCache(URL + "028", false);
OfflineTest.checkCache(URL + "029", false);
OfflineTest.checkCache(URL + "030", false);
OfflineTest.teardown();
OfflineTest.finish();
}
if (OfflineTest.setup()) {
// Wait some time after the update has been canceled to catch potential leaks of channels that would cause
// unwanted items to be cached regardless the update has been canceled with a failure.
var privUpdateCanceled = OfflineTest.priv(updateCanceled);
applicationCache.onerror = function() {window.setTimeout(privUpdateCanceled, 1000)};
// We don't expect this update to finish correctly.
applicationCache.oncached = OfflineTest.failEvent;
}
SimpleTest.waitForExplicitFinish();
</script>
</head>
<body>
</body>
</html>

76
dom/tests/mochitest/ajax/offline/test_bug744719.html Normal file
View File

@ -0,0 +1,76 @@
<html xmlns="http://www.w3.org/1999/xhtml" manifest="http://mochi.test:8888/tests/dom/tests/mochitest/ajax/offline/744719.cacheManifest">
<head>
<title>parallel load</title>
<script type="text/javascript" src="/tests/SimpleTest/SimpleTest.js"></script>
<script type="text/javascript" src="/tests/dom/tests/mochitest/ajax/offline/offlineTests.js"></script>
<link rel="stylesheet" type="text/css" href="/tests/SimpleTest/test.css" />
<script type="text/javascript">
/*
Simply load a large number of resources and check all are properly cached. This should cover all parts
of the parallel loading code.
*/
ok(applicationCache.mozItems.length == 0,
"applicationCache.mozItems should be available and empty before associating with a cache.");
function manifestUpdated()
{
OfflineTest.checkCache("http://mochi.test:8888/tests/dom/tests/mochitest/ajax/offline/744719.cacheManifest", true);
OfflineTest.checkCache("http://mochi.test:8888/tests/SimpleTest/SimpleTest.js", true);
OfflineTest.checkCache("http://mochi.test:8888/tests/dom/tests/mochitest/ajax/offline/offlineTests.js", true);
var URL = "http://mochi.test:8888/tests/dom/tests/mochitest/ajax/offline/subresource744719.html?";
OfflineTest.checkCache(URL + "001", true);
OfflineTest.checkCache(URL + "002", true);
OfflineTest.checkCache(URL + "003", true);
OfflineTest.checkCache(URL + "004", true);
OfflineTest.checkCache(URL + "005", true);
OfflineTest.checkCache(URL + "006", true);
OfflineTest.checkCache(URL + "007", true);
OfflineTest.checkCache(URL + "008", true);
OfflineTest.checkCache(URL + "009", true);
OfflineTest.checkCache(URL + "010", true);
OfflineTest.checkCache(URL + "011", true);
OfflineTest.checkCache(URL + "012", true);
OfflineTest.checkCache(URL + "013", true);
OfflineTest.checkCache(URL + "014", true);
OfflineTest.checkCache(URL + "015", true);
OfflineTest.checkCache(URL + "016", true);
OfflineTest.checkCache(URL + "017", true);
OfflineTest.checkCache(URL + "018", true);
OfflineTest.checkCache(URL + "019", true);
OfflineTest.checkCache(URL + "020", true);
OfflineTest.checkCache(URL + "021", true);
OfflineTest.checkCache(URL + "022", true);
OfflineTest.checkCache(URL + "023", true);
OfflineTest.checkCache(URL + "024", true);
OfflineTest.checkCache(URL + "025", true);
OfflineTest.checkCache(URL + "026", true);
OfflineTest.checkCache(URL + "027", true);
OfflineTest.checkCache(URL + "028", true);
OfflineTest.checkCache(URL + "029", true);
OfflineTest.checkCache(URL + "030", true);
OfflineTest.teardown();
OfflineTest.finish();
}
if (OfflineTest.setup()) {
applicationCache.onerror = OfflineTest.failEvent;
applicationCache.oncached = OfflineTest.priv(manifestUpdated);
}
SimpleTest.waitForExplicitFinish();
</script>
</head>
<body>
</body>
</html>

9
netwerk/base/public/nsIApplicationCache.idl
View File

@ -7,6 +7,7 @@
#include "nsISupports.idl"
interface nsIArray;
interface nsILocalFile;
/**
* Application caches can store a set of namespace entries that affect
@ -78,7 +79,7 @@ interface nsIApplicationCacheNamespace : nsISupports
* loads. Inactive caches will be removed from the cache when they are
* no longer referenced.
*/
[scriptable, uuid(32f83e3f-470c-4423-a86a-d35d1c215ccb)]
[scriptable, uuid(231e1e53-05c1-41b6-b9de-dbbcce9385c9)]
interface nsIApplicationCache : nsISupports
{
/**
@ -189,4 +190,10 @@ interface nsIApplicationCache : nsISupports
* Get the most specific namespace matching a given key.
*/
nsIApplicationCacheNamespace getMatchingNamespace(in ACString key);
/**
* If set, this offline cache is placed in a different directory
* than the current application profile.
*/
readonly attribute nsILocalFile cacheDirectory;
};

19
netwerk/base/public/nsIApplicationCacheService.idl
View File

@ -7,12 +7,13 @@
#include "nsISupports.idl"
interface nsIApplicationCache;
interface nsILocalFile;
/**
* The application cache service manages the set of application cache
* groups.
*/
[scriptable, uuid(10fdea21-1224-4c29-8507-8f3205a121d5)]
[scriptable, uuid(28adfdc7-6718-4b3e-bdb2-ecfefa3c8910)]
interface nsIApplicationCacheService : nsISupports
{
/**
@ -21,6 +22,22 @@ interface nsIApplicationCacheService : nsISupports
*/
nsIApplicationCache createApplicationCache(in ACString group);
/**
* Create a new, empty application cache for the given cache
* group residing in a custom directory with a custom quota.
*
* @param group
* URL of the manifest
* @param directory
* Actually a reference to a profile directory where to
* create the OfflineCache sub-dir.
* @param quota
* Optional override of the default quota.
*/
nsIApplicationCache createCustomApplicationCache(in ACString group,
in nsILocalFile profileDir,
in PRInt32 quota);
/**
* Get an application cache object for the given client ID.
*/

9
netwerk/base/public/nsICachingChannel.idl
View File

@ -6,6 +6,7 @@
#include "nsICacheInfoChannel.idl"
interface nsIFile;
interface nsILocalFile;
/**
* A channel may optionally implement this interface to allow clients
@ -17,7 +18,7 @@ interface nsIFile;
* 3) Support for uniquely identifying cached data in cases when the URL
* is insufficient (e.g., HTTP form submission).
*/
[scriptable, uuid(830D4BCB-3E46-4011-9BDA-51A5D1AF891F)]
[scriptable, uuid(E2143B61-62FE-4da5-BE2E-E31981095889)]
interface nsICachingChannel : nsICacheInfoChannel
{
/**
@ -87,6 +88,12 @@ interface nsICachingChannel : nsICacheInfoChannel
*/
attribute ACString offlineCacheClientID;
/**
* Override base (profile) directory to work with when accessing the cache.
* When not specified, the current process' profile directory will be used.
*/
attribute nsILocalFile profileDirectory;
/**
* Get the "file" where the cached data can be found. This is valid for
* as long as a reference to the cache token is held. This may return

17
netwerk/cache/nsApplicationCacheService.cpp vendored
View File

@ -34,6 +34,23 @@ nsApplicationCacheService::CreateApplicationCache(const nsACString &group,
return device->CreateApplicationCache(group, out);
}
NS_IMETHODIMP
nsApplicationCacheService::CreateCustomApplicationCache(const nsACString & group,
nsILocalFile *profileDir,
PRInt32 quota,
nsIApplicationCache **out)
{
if (!mCacheService)
return NS_ERROR_UNEXPECTED;
nsRefPtr<nsOfflineCacheDevice> device;
nsresult rv = mCacheService->GetCustomOfflineDevice(profileDir,
quota,
getter_AddRefs(device));
NS_ENSURE_SUCCESS(rv, rv);
return device->CreateApplicationCache(group, out);
}
NS_IMETHODIMP
nsApplicationCacheService::GetApplicationCache(const nsACString &clientID,
nsIApplicationCache **out)

1
netwerk/cache/nsCacheEntry.cpp vendored
View File

@ -32,6 +32,7 @@ nsCacheEntry::nsCacheEntry(nsCString * key,
mPredictedDataSize(-1),
mDataSize(0),
mCacheDevice(nsnull),
mCustomDevice(nsnull),
mData(nsnull)
{
MOZ_COUNT_CTOR(nsCacheEntry);

4
netwerk/cache/nsCacheEntry.h vendored
View File

@ -63,6 +63,9 @@ public:
nsCacheDevice * CacheDevice() { return mCacheDevice; }
void SetCacheDevice( nsCacheDevice * device) { mCacheDevice = device; }
void SetCustomCacheDevice( nsCacheDevice * device )
{ mCustomDevice = device; }
nsCacheDevice * CustomCacheDevice() { return mCustomDevice; }
const char * GetDeviceID();
/**
@ -216,6 +219,7 @@ private:
PRInt64 mPredictedDataSize; // Size given by ContentLength.
PRUint32 mDataSize; // 4
nsCacheDevice * mCacheDevice; // 4
nsCacheDevice * mCustomDevice; // 4
nsCOMPtr<nsISupports> mSecurityInfo; //
nsISupports * mData; // strong ref
nsCOMPtr<nsIThread> mThread;

4
netwerk/cache/nsCacheRequest.h vendored
View File

@ -37,7 +37,8 @@ private:
mInfo(0),
mListener(listener),
mLock("nsCacheRequest.mLock"),
mCondVar(mLock, "nsCacheRequest.mCondVar")
mCondVar(mLock, "nsCacheRequest.mCondVar"),
mProfileDir(session->ProfileDir())
{
MOZ_COUNT_CTOR(nsCacheRequest);
PR_INIT_CLIST(this);
@ -152,6 +153,7 @@ private:
nsCOMPtr<nsIThread> mThread;
Mutex mLock;
CondVar mCondVar;
nsCOMPtr<nsILocalFile> mProfileDir;
};
#endif // _nsCacheRequest_h_

108
netwerk/cache/nsCacheService.cpp vendored
View File

@ -1089,6 +1089,7 @@ nsCacheService::nsCacheService()
// create list of cache devices
PR_INIT_CLIST(&mDoomedEntries);
mCustomOfflineDevices.Init();
}
nsCacheService::~nsCacheService()
@ -1143,6 +1144,15 @@ nsCacheService::Init()
return NS_OK;
}
// static
PLDHashOperator
nsCacheService::ShutdownCustomCacheDeviceEnum(const nsAString& aProfileDir,
nsRefPtr<nsOfflineCacheDevice>& aDevice,
void* aUserArg)
{
aDevice->Shutdown();
return PL_DHASH_REMOVE;
}
void
nsCacheService::Shutdown()
@ -1198,6 +1208,8 @@ nsCacheService::Shutdown()
NS_IF_RELEASE(mOfflineDevice);
mCustomOfflineDevices.Enumerate(&nsCacheService::ShutdownCustomCacheDeviceEnum, nsnull);
#ifdef PR_LOGGING
LogCacheStatistics();
#endif
@ -1532,32 +1544,75 @@ nsCacheService::GetOfflineDevice(nsOfflineCacheDevice **aDevice)
return NS_OK;
}
nsresult
nsCacheService::GetCustomOfflineDevice(nsILocalFile *aProfileDir,
PRInt32 aQuota,
nsOfflineCacheDevice **aDevice)
{
nsresult rv;
nsAutoString profilePath;
rv = aProfileDir->GetPath(profilePath);
NS_ENSURE_SUCCESS(rv, rv);
if (!mCustomOfflineDevices.Get(profilePath, aDevice)) {
rv = CreateCustomOfflineDevice(aProfileDir, aQuota, aDevice);
NS_ENSURE_SUCCESS(rv, rv);
mCustomOfflineDevices.Put(profilePath, *aDevice);
}
return NS_OK;
}
nsresult
nsCacheService::CreateOfflineDevice()
{
CACHE_LOG_ALWAYS(("Creating offline device"));
CACHE_LOG_ALWAYS(("Creating default offline device"));
if (mOfflineDevice) return NS_OK;
if (!mObserver) return NS_ERROR_NOT_AVAILABLE;
nsresult rv = CreateCustomOfflineDevice(
mObserver->OfflineCacheParentDirectory(),
mObserver->OfflineCacheCapacity(),
&mOfflineDevice);
NS_ENSURE_SUCCESS(rv, rv);
return NS_OK;
}
nsresult
nsCacheService::CreateCustomOfflineDevice(nsILocalFile *aProfileDir,
PRInt32 aQuota,
nsOfflineCacheDevice **aDevice)
{
NS_ENSURE_ARG(aProfileDir);
#if defined(PR_LOGGING)
nsCAutoString profilePath;
aProfileDir->GetNativePath(profilePath);
CACHE_LOG_ALWAYS(("Creating custom offline device, %s, %d",
profilePath.BeginReading(), aQuota));
#endif
if (!mInitialized) return NS_ERROR_NOT_AVAILABLE;
if (!mEnableOfflineDevice) return NS_ERROR_NOT_AVAILABLE;
if (mOfflineDevice) return NS_OK;
mOfflineDevice = new nsOfflineCacheDevice;
if (!mOfflineDevice) return NS_ERROR_OUT_OF_MEMORY;
*aDevice = new nsOfflineCacheDevice;
NS_ADDREF(mOfflineDevice);
NS_ADDREF(*aDevice);
// set the preferences
mOfflineDevice->SetCacheParentDirectory(
mObserver->OfflineCacheParentDirectory());
mOfflineDevice->SetCapacity(mObserver->OfflineCacheCapacity());
(*aDevice)->SetCacheParentDirectory(aProfileDir);
(*aDevice)->SetCapacity(aQuota);
nsresult rv = mOfflineDevice->Init();
nsresult rv = (*aDevice)->Init();
if (NS_FAILED(rv)) {
CACHE_LOG_DEBUG(("mOfflineDevice->Init() failed (0x%.8x)\n", rv));
CACHE_LOG_DEBUG(("OfflineDevice->Init() failed (0x%.8x)\n", rv));
CACHE_LOG_DEBUG((" - disabling offline cache for this session.\n"));
mEnableOfflineDevice = false;
NS_RELEASE(mOfflineDevice);
NS_RELEASE(*aDevice);
}
return rv;
}
@ -1734,6 +1789,20 @@ nsCacheService::ProcessRequest(nsCacheRequest * request,
// loop back around to look for another entry
}
if (NS_SUCCEEDED(rv) && request->mProfileDir) {
// Custom cache directory has been demanded. Preset the cache device.
if (entry->StoragePolicy() != nsICache::STORE_OFFLINE) {
// Failsafe check: this is implemented only for offline cache atm.
rv = NS_ERROR_FAILURE;
} else {
nsRefPtr<nsOfflineCacheDevice> customCacheDevice;
rv = GetCustomOfflineDevice(request->mProfileDir, -1,
getter_AddRefs(customCacheDevice));
if (NS_SUCCEEDED(rv))
entry->SetCustomCacheDevice(customCacheDevice);
}
}
nsICacheEntryDescriptor *descriptor = nsnull;
if (NS_SUCCEEDED(rv))
@ -2046,12 +2115,16 @@ nsCacheService::EnsureEntryHasDevice(nsCacheEntry * entry)
(void)CreateOfflineDevice(); // ignore the error (check for mOfflineDevice instead)
}
if (mOfflineDevice) {
device = entry->CustomCacheDevice()
? entry->CustomCacheDevice()
: mOfflineDevice;
if (device) {
entry->MarkBinding();
nsresult rv = mOfflineDevice->BindEntry(entry);
nsresult rv = device->BindEntry(entry);
entry->ClearBinding();
if (NS_SUCCEEDED(rv))
device = mOfflineDevice;
if (NS_FAILED(rv))
device = nsnull;
}
}
@ -2146,6 +2219,9 @@ nsCacheService::OnProfileShutdown(bool cleanse)
gService->mOfflineDevice->Shutdown();
}
gService->mCustomOfflineDevices.Enumerate(
&nsCacheService::ShutdownCustomCacheDeviceEnum, nsnull);
gService->mEnableOfflineDevice = false;
if (gService->mMemoryDevice) {

20
netwerk/cache/nsCacheService.h vendored
View File

@ -17,6 +17,7 @@
#include "nsIObserver.h"
#include "nsString.h"
#include "nsTArray.h"
#include "nsRefPtrHashtable.h"
#include "mozilla/CondVar.h"
#include "mozilla/Mutex.h"
@ -112,6 +113,15 @@ public:
nsresult GetOfflineDevice(nsOfflineCacheDevice ** aDevice);
/**
* Creates an offline cache device that works over a specific profile directory.
* A tool to preload offline cache for profiles different from the current
* application's profile directory.
*/
nsresult GetCustomOfflineDevice(nsILocalFile *aProfileDir,
PRInt32 aQuota,
nsOfflineCacheDevice **aDevice);
// This method may be called to release an object while the cache service
// lock is being held. If a non-null target is specified and the target
// does not correspond to the current thread, then the release will be
@ -183,6 +193,9 @@ private:
nsresult CreateDiskDevice();
nsresult CreateOfflineDevice();
nsresult CreateCustomOfflineDevice(nsILocalFile *aProfileDir,
PRInt32 aQuota,
nsOfflineCacheDevice **aDevice);
nsresult CreateMemoryDevice();
nsresult CreateRequest(nsCacheSession * session,
@ -237,6 +250,11 @@ private:
PLDHashEntryHdr * hdr,
PRUint32 number,
void * arg);
static
PLDHashOperator ShutdownCustomCacheDeviceEnum(const nsAString& aProfileDir,
nsRefPtr<nsOfflineCacheDevice>& aDevice,
void* aUserArg);
#if defined(PR_LOGGING)
void LogCacheStatistics();
#endif
@ -270,6 +288,8 @@ private:
nsDiskCacheDevice * mDiskDevice;
nsOfflineCacheDevice * mOfflineDevice;
nsRefPtrHashtable<nsStringHashKey, nsOfflineCacheDevice> mCustomOfflineDevices;
nsCacheEntryHashTable mActiveEntries;
PRCList mDoomedEntries;

25
netwerk/cache/nsCacheSession.cpp vendored
View File

@ -41,6 +41,31 @@ NS_IMETHODIMP nsCacheSession::GetDoomEntriesIfExpired(bool *result)
}
NS_IMETHODIMP nsCacheSession::SetProfileDirectory(nsILocalFile *profileDir)
{
if (StoragePolicy() != nsICache::STORE_OFFLINE && profileDir) {
// Profile directory override is currently implemented only for
// offline cache. This is an early failure to prevent the request
// being processed before it would fail later because of inability
// to assign a cache base dir.
return NS_ERROR_UNEXPECTED;
}
mProfileDir = profileDir;
return NS_OK;
}
NS_IMETHODIMP nsCacheSession::GetProfileDirectory(nsILocalFile **profileDir)
{
if (mProfileDir)
NS_ADDREF(*profileDir = mProfileDir);
else
*profileDir = nsnull;
return NS_OK;
}
NS_IMETHODIMP nsCacheSession::SetDoomEntriesIfExpired(bool doomEntriesIfExpired)
{
if (doomEntriesIfExpired) MarkDoomEntriesIfExpired();

5
netwerk/cache/nsCacheSession.h vendored
View File

@ -9,7 +9,9 @@
#include "nspr.h"
#include "nsError.h"
#include "nsCOMPtr.h"
#include "nsICacheSession.h"
#include "nsILocalFile.h"
#include "nsString.h"
class nsCacheSession : public nsICacheSession
@ -53,9 +55,12 @@ public:
mInfo |= policy;
}
nsILocalFile* ProfileDir() { return mProfileDir; }
private:
nsCString mClientID;
PRUint32 mInfo;
nsCOMPtr<nsILocalFile> mProfileDir;
};
#endif // _nsCacheSession_h_

13
netwerk/cache/nsDiskCacheDeviceSQL.cpp vendored
View File

@ -629,6 +629,17 @@ nsApplicationCache::GetClientID(nsACString &out)
return NS_OK;
}
NS_IMETHODIMP
nsApplicationCache::GetCacheDirectory(nsILocalFile **out)
{
if (mDevice->BaseDirectory())
NS_ADDREF(*out = mDevice->BaseDirectory());
else
*out = nsnull;
return NS_OK;
}
NS_IMETHODIMP
nsApplicationCache::GetActive(bool *out)
{
@ -2361,6 +2372,8 @@ nsOfflineCacheDevice::SetCacheParentDirectory(nsILocalFile *parentDir)
return;
}
mBaseDirectory = parentDir;
// cache dir may not exist, but that's ok
nsCOMPtr<nsIFile> dir;
rv = parentDir->Clone(getter_AddRefs(dir));

2
netwerk/cache/nsDiskCacheDeviceSQL.h vendored
View File

@ -168,6 +168,7 @@ public:
void SetCacheParentDirectory(nsILocalFile * parentDir);
void SetCapacity(PRUint32 capacity);
nsILocalFile * BaseDirectory() { return mBaseDirectory; }
nsILocalFile * CacheDirectory() { return mCacheDirectory; }
PRUint32 CacheCapacity() { return mCacheCapacity; }
PRUint32 CacheSize();
@ -252,6 +253,7 @@ private:
nsCOMPtr<mozIStorageStatement> mStatement_EnumerateGroups;
nsCOMPtr<mozIStorageStatement> mStatement_EnumerateGroupsTimeOrder;
nsCOMPtr<nsILocalFile> mBaseDirectory;
nsCOMPtr<nsILocalFile> mCacheDirectory;
PRUint32 mCacheCapacity; // in bytes
PRInt32 mDeltaCounter;

9
netwerk/cache/nsICacheSession.idl vendored
View File

@ -9,6 +9,7 @@
interface nsICacheEntryDescriptor;
interface nsICacheListener;
interface nsILocalFile;
[scriptable, uuid(1dd7708c-de48-4ffe-b5aa-cd218c762887)]
interface nsICacheSession : nsISupports
@ -21,6 +22,14 @@ interface nsICacheSession : nsISupports
*/
attribute boolean doomEntriesIfExpired;
/**
* When set, entries created with this session will be placed to a cache
* based at this directory. Use when storing entries to a different
* profile than the active profile of the the current running application
* process.
*/
attribute nsILocalFile profileDirectory;
/**
* A cache session can only give out one descriptor with WRITE access
* to a given cache entry at a time. Until the client calls MarkValid on

22
netwerk/protocol/http/nsHttpChannel.cpp
View File

@ -2617,6 +2617,11 @@ nsHttpChannel::OpenOfflineCacheEntryForWriting()
getter_AddRefs(session));
if (NS_FAILED(rv)) return rv;
if (mProfileDirectory) {
rv = session->SetProfileDirectory(mProfileDirectory);
if (NS_FAILED(rv)) return rv;
}
mOnCacheEntryAvailableCallback =
&nsHttpChannel::OnOfflineCacheEntryForWritingAvailable;
rv = session->AsyncOpenCacheEntry(cacheKey, nsICache::ACCESS_READ_WRITE,
@ -3934,6 +3939,7 @@ nsHttpChannel::SetupReplacementChannel(nsIURI *newURI,
// cacheClientID, cacheForOfflineUse
cachingChannel->SetOfflineCacheClientID(mOfflineCacheClientID);
cachingChannel->SetCacheForOfflineUse(mCacheForOfflineUse);
cachingChannel->SetProfileDirectory(mProfileDirectory);
}
}
@ -5361,6 +5367,22 @@ nsHttpChannel::SetOfflineCacheClientID(const nsACString &value)
return NS_OK;
}
NS_IMETHODIMP
nsHttpChannel::GetProfileDirectory(nsILocalFile **_result)
{
NS_ENSURE_ARG(_result);
NS_ADDREF(*_result = mProfileDirectory);
return NS_OK;
}
NS_IMETHODIMP
nsHttpChannel::SetProfileDirectory(nsILocalFile *value)
{
mProfileDirectory = value;
return NS_OK;
}
NS_IMETHODIMP
nsHttpChannel::GetCacheFile(nsIFile **cacheFile)
{

3
netwerk/protocol/http/nsHttpChannel.h
View File

@ -27,6 +27,7 @@
#include "nsIHttpChannelAuthProvider.h"
#include "nsIAsyncVerifyRedirectCallback.h"
#include "nsITimedChannel.h"
#include "nsILocalFile.h"
#include "nsDNSPrefetch.h"
#include "TimingStruct.h"
#include "AutoClose.h"
@ -304,6 +305,8 @@ private:
nsCacheAccessMode mOfflineCacheAccess;
nsCString mOfflineCacheClientID;
nsCOMPtr<nsILocalFile> mProfileDirectory;
// auth specific data
nsCOMPtr<nsIHttpChannelAuthProvider> mAuthProvider;

125
netwerk/test/unit/test_offlinecache_custom-directory.js Normal file
View File

@ -0,0 +1,125 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this file,
* You can obtain one at http://mozilla.org/MPL/2.0/. */
/**
* This test executes nsIOfflineCacheUpdateService.scheduleCustomProfileUpdate API
* 1. preloads an app with a manifest to a custom sudir in the profile (for simplicity)
* 2. observes progress and completion of the update
* 3. checks presence of index.sql and files in the expected location
*/
do_load_httpd_js();
var httpServer = null;
var cacheUpdateObserver = null;
function make_channel(url, callback, ctx) {
var ios = Cc["@mozilla.org/network/io-service;1"].
getService(Ci.nsIIOService);
return ios.newChannel(url, "", null);
}
function make_uri(url) {
var ios = Cc["@mozilla.org/network/io-service;1"].
getService(Ci.nsIIOService);
return ios.newURI(url, null, null);
}
// start the test with loading this master entry referencing the manifest
function masterEntryHandler(metadata, response)
{
var masterEntryContent = "<html manifest='/manifest'></html>";
response.setHeader("Content-Type", "text/html");
response.bodyOutputStream.write(masterEntryContent, masterEntryContent.length);
}
// manifest defines fallback namespace from any /redirect path to /content
function manifestHandler(metadata, response)
{
var manifestContent = "CACHE MANIFEST\n";
response.setHeader("Content-Type", "text/cache-manifest");
response.bodyOutputStream.write(manifestContent, manifestContent.length);
}
// finally check we got fallback content
function finish_test(customDir)
{
var offlineCacheDir = customDir.clone();
offlineCacheDir.append("OfflineCache");
var indexSqlFile = offlineCacheDir.clone();
indexSqlFile.append('index.sqlite');
do_check_eq(indexSqlFile.exists(), true);
var file1 = offlineCacheDir.clone();
file1.append("2");
file1.append("E");
file1.append("2C99DE6E7289A5-0");
do_check_eq(file1.exists(), true);
var file2 = offlineCacheDir.clone();
file2.append("8");
file2.append("6");
file2.append("0B457F75198B29-0");
do_check_eq(file2.exists(), true);
httpServer.stop(do_test_finished);
}
function run_test()
{
httpServer = new nsHttpServer();
httpServer.registerPathHandler("/masterEntry", masterEntryHandler);
httpServer.registerPathHandler("/manifest", manifestHandler);
httpServer.start(4444);
var profileDir = do_get_profile();
var customDir = profileDir.clone();
customDir.append("customOfflineCacheDir" + Math.random());
var pm = Cc["@mozilla.org/permissionmanager;1"]
.getService(Ci.nsIPermissionManager);
var uri = make_uri("http://localhost:4444");
if (pm.testPermission(uri, "offline-app") != 0) {
dump("Previous test failed to clear offline-app permission! Expect failures.\n");
}
pm.add(uri, "offline-app", Ci.nsIPermissionManager.ALLOW_ACTION);
var ps = Cc["@mozilla.org/preferences-service;1"]
.getService(Ci.nsIPrefBranch);
ps.setBoolPref("browser.cache.offline.enable", true);
// Set this pref to mimic the default browser behavior.
ps.setComplexValue("browser.cache.offline.parent_directory", Ci.nsILocalFile, profileDir);
var us = Cc["@mozilla.org/offlinecacheupdate-service;1"].
getService(Ci.nsIOfflineCacheUpdateService);
var update = us.scheduleCustomProfileUpdate(
make_uri("http://localhost:4444/manifest"),
make_uri("http://localhost:4444/masterEntry"),
customDir);
var expectedStates = [
Ci.nsIOfflineCacheUpdateObserver.STATE_DOWNLOADING,
Ci.nsIOfflineCacheUpdateObserver.STATE_ITEMSTARTED,
Ci.nsIOfflineCacheUpdateObserver.STATE_ITEMPROGRESS,
Ci.nsIOfflineCacheUpdateObserver.STATE_ITEMCOMPLETED,
Ci.nsIOfflineCacheUpdateObserver.STATE_FINISHED
];
update.addObserver({
updateStateChanged: function(update, state)
{
do_check_eq(state, expectedStates.shift());
if (state == Ci.nsIOfflineCacheUpdateObserver.STATE_FINISHED)
finish_test(customDir);
},
applicationCacheAvailable: function(appCache)
{
}
}, false);
do_test_pending();
}

1
netwerk/test/unit/xpcshell.ini
View File

@ -187,3 +187,4 @@ run-if = hasNode
[test_xmlhttprequest.js]
[test_XHR_redirects.js]
[test_pinned_app_cache.js]
[test_offlinecache_custom-directory.js]

1022
other-licenses/virtualenv/virtualenv.egg-info/PKG-INFO
View File

File diff suppressed because it is too large Load Diff

33
other-licenses/virtualenv/virtualenv.egg-info/SOURCES.txt
View File

@ -1,33 +0,0 @@
AUTHORS.txt
LICENSE.txt
MANIFEST.in
setup.py
virtualenv.py
docs/index.txt
docs/news.txt
scripts/virtualenv
virtualenv.egg-info/PKG-INFO
virtualenv.egg-info/SOURCES.txt
virtualenv.egg-info/dependency_links.txt
virtualenv.egg-info/entry_points.txt
virtualenv.egg-info/not-zip-safe
virtualenv.egg-info/top_level.txt
virtualenv_embedded/activate.bat
virtualenv_embedded/activate.csh
virtualenv_embedded/activate.fish
virtualenv_embedded/activate.ps1
virtualenv_embedded/activate.sh
virtualenv_embedded/activate_this.py
virtualenv_embedded/deactivate.bat
virtualenv_embedded/distribute_setup.py
virtualenv_embedded/distutils-init.py
virtualenv_embedded/distutils.cfg
virtualenv_embedded/ez_setup.py
virtualenv_embedded/site.py
virtualenv_support/__init__.py
virtualenv_support/distribute-0.6.24.tar.gz
virtualenv_support/pip-1.1.tar.gz
virtualenv_support/setuptools-0.6c11-py2.4.egg
virtualenv_support/setuptools-0.6c11-py2.5.egg
virtualenv_support/setuptools-0.6c11-py2.6.egg
virtualenv_support/setuptools-0.6c11-py2.7.egg

1
other-licenses/virtualenv/virtualenv.egg-info/dependency_links.txt
View File

@ -1 +0,0 @@

2
other-licenses/virtualenv/virtualenv.egg-info/entry_points.txt
View File

@ -1,2 +0,0 @@
[console_scripts]
virtualenv = virtualenv:main

1
other-licenses/virtualenv/virtualenv.egg-info/not-zip-safe
View File

@ -1 +0,0 @@

2
other-licenses/virtualenv/virtualenv.egg-info/top_level.txt
View File

@ -1,2 +0,0 @@
virtualenv_support
virtualenv

2
services/Makefile.in
View File

@ -11,7 +11,7 @@ VPATH = @srcdir@
include $(DEPTH)/config/autoconf.mk
ifdef MOZ_SERVICES_SYNC
PARALLEL_DIRS += common crypto sync
PARALLEL_DIRS += aitc common crypto sync
endif
include $(topsrcdir)/config/rules.mk

121
services/aitc/Aitc.js Normal file
View File

@ -0,0 +1,121 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this file,
* You can obtain one at http://mozilla.org/MPL/2.0/. */
"use strict";
const {classes: Cc, interfaces: Ci, utils: Cu, results: Cr} = Components;
Cu.import("resource://gre/modules/XPCOMUtils.jsm");
Cu.import("resource://gre/modules/Services.jsm");
Cu.import("resource://gre/modules/PlacesUtils.jsm");
Cu.import("resource://services-common/utils.js");
function AitcService() {
this.aitc = null;
this.wrappedJSObject = this;
}
AitcService.prototype = {
classID: Components.ID("{a3d387ca-fd26-44ca-93be-adb5fda5a78d}"),
QueryInterface: XPCOMUtils.generateQI([Ci.nsIObserver,
Ci.nsINavHistoryObserver,
Ci.nsISupportsWeakReference]),
observe: function observe(subject, topic, data) {
switch (topic) {
case "app-startup":
// We listen for this event beacause Aitc won't work until there is
// atleast 1 visible top-level XUL window.
Services.obs.addObserver(this, "sessionstore-windows-restored", true);
break;
case "sessionstore-windows-restored":
Services.obs.removeObserver(this, "sessionstore-windows-restored");
// Don't start AITC if classic sync is on.
Cu.import("resource://services-common/preferences.js");
if (Preferences.get("services.sync.engine.apps", false)) {
return;
}
// Start AITC only if it is enabled.
if (!Preferences.get("services.aitc.enabled", false)) {
return;
}
// Start AITC service if apps.enabled is true. If false, we look
// in the browser history to determine if they're an "apps user". If
// an entry wasn't found, we'll watch for navigation to either the
// marketplace or dashboard and switch ourselves on then.
if (Preferences.get("apps.enabled", false)) {
this.start();
return;
}
// Set commonly used URLs.
this.DASHBOARD_URL = CommonUtils.makeURI(
Preferences.get("services.aitc.dashboard.url")
);
this.MARKETPLACE_URL = CommonUtils.makeURI(
Preferences.get("services.aitc.marketplace.url")
);
if (this.hasUsedApps()) {
Preferences.set("apps.enabled", true);
this.start();
return;
}
// Wait and see if the user wants anything apps related.
PlacesUtils.history.addObserver(this, true);
break;
}
},
start: function start() {
Cu.import("resource://services-aitc/main.js");
if (!this.aitc) {
this.aitc = new Aitc();
}
},
hasUsedApps: function hasUsedApps() {
// There is no easy way to determine whether a user is "using apps".
// The best we can do right now is to see if they have visited either
// the Mozilla dashboard or Marketplace. See bug 760898.
let gh = PlacesUtils.ghistory2;
if (gh.isVisited(this.DASHBOARD_URL)) {
return true;
}
if (gh.isVisited(this.MARKETPLACE_URL)) {
return true;
}
return false;
},
// nsINavHistoryObserver. We are only interested in onVisit().
onBeforeDeleteURI: function() {},
onBeginUpdateBatch: function() {},
onClearHistory: function() {},
onDeleteURI: function() {},
onDeleteVisits: function() {},
onEndUpdateBatch: function() {},
onPageChanged: function() {},
onPageExpired: function() {},
onTitleChanged: function() {},
onVisit: function onVisit(uri) {
if (!uri.equals(this.MARKETPLACE_URL) && !uri.equals(this.DASHBOARD_URL)) {
return;
}
PlacesUtils.history.removeObserver(this);
Preferences.set("apps.enabled", true);
this.start();
return;
},
};
const components = [AitcService];
const NSGetFactory = XPCOMUtils.generateNSGetFactory(components);

6
services/aitc/AitcComponents.manifest Normal file
View File

@ -0,0 +1,6 @@
# Aitc.js
component {a3d387ca-fd26-44ca-93be-adb5fda5a78d} Aitc.js
contract @mozilla.org/services/aitc;1 {a3d387ca-fd26-44ca-93be-adb5fda5a78d}
category app-startup AitcService service,@mozilla.org/services/aitc;1
# Register resource aliases
resource services-aitc resource:///modules/services-aitc/

24
services/aitc/Makefile.in Normal file
View File

@ -0,0 +1,24 @@
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this file,
# You can obtain one at http://mozilla.org/MPL/2.0/.
DEPTH = ../..
topsrcdir = @top_srcdir@
srcdir = @srcdir@
VPATH = @srcdir@
include $(DEPTH)/config/autoconf.mk
EXTRA_COMPONENTS = \
AitcComponents.manifest \
Aitc.js \
$(NULL)
PREF_JS_EXPORTS = $(srcdir)/services-aitc.js
libs::
$(NSINSTALL) $(srcdir)/modules/* $(FINAL_TARGET)/modules/services-aitc
TEST_DIRS += tests
include $(topsrcdir)/config/rules.mk

458
services/aitc/modules/browserid.js Normal file
View File

@ -0,0 +1,458 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this file,
* You can obtain one at http://mozilla.org/MPL/2.0/. */
"use strict";
const EXPORTED_SYMBOLS = ["BrowserID"];
const {classes: Cc, interfaces: Ci, utils: Cu} = Components;
Cu.import("resource://gre/modules/Services.jsm");
Cu.import("resource://gre/modules/XPCOMUtils.jsm");
Cu.import("resource://services-common/utils.js");
Cu.import("resource://services-common/log4moz.js");
Cu.import("resource://services-common/preferences.js");
const PREFS = new Preferences("services.aitc.browserid.");
/**
* This implementation will be replaced with native crypto and assertion
* generation goodness. See bug 753238.
*/
function BrowserIDService() {
this._log = Log4Moz.repository.getLogger("Services.BrowserID");
this._log.level = Log4Moz.Level[PREFS.get("log")];
}
BrowserIDService.prototype = {
/**
* Getter that returns the freshest value for ID_URI.
*/
get ID_URI() {
return PREFS.get("url");
},
/**
* Obtain a BrowserID assertion with the specified characteristics.
*
* @param cb
* (Function) Callback to be called with (err, assertion) where 'err'
* can be an Error or NULL, and 'assertion' can be NULL or a valid
* BrowserID assertion. If no callback is provided, an exception is
* thrown.
*
* @param options
* (Object) An object that may contain the following properties:
*
* "requiredEmail" : An email for which the assertion is to be
* issued. If one could not be obtained, the call
* will fail. If this property is not specified,
* the default email as set by the user will be
* chosen. If both this property and "sameEmailAs"
* are set, an exception will be thrown.
*
* "sameEmailAs" : If set, instructs the function to issue an
* assertion for the same email that was provided
* to the domain specified by this value. If this
* information could not be obtained, the call
* will fail. If both this property and
* "requiredEmail" are set, an exception will be
* thrown.
*
* "audience" : The audience for which the assertion is to be
* issued. If this property is not set an exception
* will be thrown.
*
* Any properties not listed above will be ignored.
*
* (This function could use some love in terms of what arguments it accepts.
* See bug 746401.)
*/
getAssertion: function getAssertion(cb, options) {
if (!cb) {
throw new Error("getAssertion called without a callback");
}
if (!options) {
throw new Error("getAssertion called without any options");
}
if (!options.audience) {
throw new Error("getAssertion called without an audience");
}
if (options.sameEmailAs && options.requiredEmail) {
throw new Error(
"getAssertion sameEmailAs and requiredEmail are mutually exclusive"
);
}
new Sandbox(this._getEmails.bind(this, cb, options), this.ID_URI);
},
/**
* Obtain a BrowserID assertion by asking the user to login and select an
* email address.
*
* @param cb
* (Function) Callback to be called with (err, assertion) where 'err'
* can be an Error or NULL, and 'assertion' can be NULL or a valid
* BrowserID assertion. If no callback is provided, an exception is
* thrown.
*
* @param win
* (Window) A contentWindow that has a valid document loaded. If this
* argument is provided the user will be asked to login in the context
* of the document currently loaded in this window.
*
* The audience of the assertion will be set to the domain of the
* loaded document, and the "audience" property in the "options"
* argument (if provided), will be ignored. The email to which this
* assertion issued will be selected by the user when they login (and
* "requiredEmail" or "sameEmailAs", if provided, will be ignored). If
* the user chooses to not login, this call will fail.
*
* Be aware! The provided contentWindow must also have loaded the
* BrowserID include.js shim for this to work! This behavior is
* temporary until we implement native support for navigator.id.
*
* @param options
* (Object) Currently an empty object. Present for future compatiblity
* when options for a login case may be added. Any properties, if
* present, are ignored.
*/
getAssertionWithLogin: function getAssertionWithLogin(cb, win, options) {
if (!cb) {
throw new Error("getAssertionWithLogin called without a callback");
}
if (!win) {
throw new Error("getAssertionWithLogin called without a window");
}
this._getAssertionWithLogin(cb, win);
},
/**
* Internal implementation methods begin here
*/
// Try to get the user's email(s). If user isn't logged in, this will be empty
_getEmails: function _getEmails(cb, options, sandbox) {
let self = this;
function callback(res) {
let emails = {};
try {
emails = JSON.parse(res);
} catch (e) {
self._log.error("Exception in JSON.parse for _getAssertion: " + e);
}
self._gotEmails(emails, sandbox, cb, options);
}
sandbox.box.importFunction(callback);
let scriptText =
"var list = window.BrowserID.User.getStoredEmailKeypairs();" +
"callback(JSON.stringify(list));";
Cu.evalInSandbox(scriptText, sandbox.box, "1.8", this.ID_URI, 1);
},
// Received a list of emails from BrowserID for current user
_gotEmails: function _gotEmails(emails, sandbox, cb, options) {
let keys = Object.keys(emails);
// If list is empty, user is not logged in, or doesn't have a default email.
if (!keys.length) {
sandbox.free();
let err = "User is not logged in, or no emails were found";
this._log.error(err);
try {
cb(new Error(err), null);
} catch(e) {
this._log.warn("Callback threw in _gotEmails " +
CommonUtils.exceptionStr(e));
}
return;
}
// User is logged in. For which email shall we get an assertion?
// Case 1: Explicitely provided
if (options.requiredEmail) {
this._getAssertionWithEmail(
sandbox, cb, options.requiredEmail, options.audience
);
return;
}
// Case 2: Derive from a given domain
if (options.sameEmailAs) {
this._getAssertionWithDomain(
sandbox, cb, options.sameEmailAs, options.audience
);
return;
}
// Case 3: Default email
this._getAssertionWithEmail(
sandbox, cb, keys[0], options.audience
);
return;
},
/**
* Open a login window and ask the user to login, returning the assertion
* generated as a result to the caller.
*/
_getAssertionWithLogin: function _getAssertionWithLogin(cb, win) {
// We're executing navigator.id.get as a content script in win.
// This results in a popup that we will temporarily unblock.
let pm = Services.perms;
let origin = Services.io.newURI(
win.wrappedJSObject.location.toString(), null, null
);
let oldPerm = pm.testExactPermission(origin, "popup");
try {
pm.add(origin, "popup", pm.ALLOW_ACTION);
} catch(e) {
this._log.warn("Setting popup blocking to false failed " + e);
}
// Open sandbox and execute script. This sandbox will be GC'ed.
let sandbox = new Cu.Sandbox(win, {
wantXrays: false,
sandboxPrototype: win
});
let self = this;
function callback(val) {
// Set popup blocker permission to original value.
try {
pm.add(origin, "popup", oldPerm);
} catch(e) {
this._log.warn("Setting popup blocking to original value failed " + e);
}
if (val) {
self._log.info("_getAssertionWithLogin succeeded");
try {
cb(null, val);
} catch(e) {
self._log.warn("Callback threw in _getAssertionWithLogin " +
CommonUtils.exceptionStr(e));
}
} else {
let msg = "Could not obtain assertion in _getAssertionWithLogin";
self._log.error(msg);
try {
cb(new Error(msg), null);
} catch(e) {
self._log.warn("Callback threw in _getAssertionWithLogin " +
CommonUtils.exceptionStr(e));
}
}
}
sandbox.importFunction(callback);
function doGetAssertion() {
self._log.info("_getAssertionWithLogin Started");
let scriptText = "window.navigator.id.get(" +
" callback, {allowPersistent: true}" +
");";
Cu.evalInSandbox(scriptText, sandbox, "1.8", self.ID_URI, 1);
}
// Sometimes the provided win hasn't fully loaded yet
if (!win.document || (win.document.readyState != "complete")) {
win.addEventListener("DOMContentLoaded", function _contentLoaded() {
win.removeEventListener("DOMContentLoaded", _contentLoaded, false);
doGetAssertion();
}, false);
} else {
doGetAssertion();
}
},
/**
* Gets an assertion for the specified 'email' and 'audience'
*/
_getAssertionWithEmail: function _getAssertionWithEmail(sandbox, cb, email,
audience) {
let self = this;
function onSuccess(res) {
// Cleanup first.
sandbox.free();
// The internal API sometimes calls onSuccess even though no assertion
// could be obtained! Double check:
if (!res) {
let msg = "BrowserID.User.getAssertion empty assertion for " + email;
self._log.error(msg);
try {
cb(new Error(msg), null);
} catch(e) {
self._log.warn("Callback threw in _getAssertionWithEmail " +
CommonUtils.exceptionStr(e));
}
return;
}
// Success
self._log.info("BrowserID.User.getAssertion succeeded");
try {
cb(null, res);
} catch(e) {
self._log.warn("Callback threw in _getAssertionWithEmail " +
CommonUtils.exceptionStr(e));
}
}
function onError(err) {
sandbox.free();
self._log.info("BrowserID.User.getAssertion failed");
try {
cb(err, null);
} catch(e) {
self._log.warn("Callback threw in _getAssertionWithEmail " +
CommonUtils.exceptionStr(e));
}
}
sandbox.box.importFunction(onSuccess);
sandbox.box.importFunction(onError);
self._log.info("_getAssertionWithEmail Started");
let scriptText =
"window.BrowserID.User.getAssertion(" +
"'" + email + "', " +
"'" + audience + "', " +
"onSuccess, " +
"onError" +
");";
Cu.evalInSandbox(scriptText, sandbox.box, "1.8", this.ID_URI, 1);
},
/**
* Gets the email which was used to login to 'domain'. If one was found,
* _getAssertionWithEmail is called to obtain the assertion.
*/
_getAssertionWithDomain: function _getAssertionWithDomain(sandbox, cb, domain,
audience) {
let self = this;
function onDomainSuccess(email) {
if (email) {
self._getAssertionWithEmail(sandbox, cb, email, audience);
} else {
sandbox.free();
try {
cb(new Error("No email found for _getAssertionWithDomain"), null);
} catch(e) {
self._log.warn("Callback threw in _getAssertionWithDomain " +
CommonUtils.exceptionStr(e));
}
}
}
sandbox.box.importFunction(onDomainSuccess);
// This wil tell us which email was used to login to "domain", if any.
self._log.info("_getAssertionWithDomain Started");
let scriptText =
"onDomainSuccess(window.BrowserID.Storage.site.get(" +
"'" + domain + "', " +
"'email'" +
"));";
Cu.evalInSandbox(scriptText, sandbox.box, "1.8", this.ID_URI, 1);
},
};
/**
* An object that represents a sandbox in an iframe loaded with uri. The
* callback provided to the constructor will be invoked when the sandbox is
* ready to be used. The callback will receive this object as its only argument
* and the prepared sandbox may be accessed via the "box" property.
*
* Please call free() when you are finished with the sandbox to explicitely free
* up all associated resources.
*
* @param cb
* (function) Callback to be invoked with a Sandbox, when ready.
* @param uri
* (String) URI to be loaded in the Sandbox.
*/
function Sandbox(cb, uri) {
this._uri = uri;
this._createFrame();
this._createSandbox(cb, uri);
}
Sandbox.prototype = {
/**
* Frees the sandbox and releases the iframe created to host it.
*/
free: function free() {
delete this.box;
this._container.removeChild(this._frame);
this._frame = null;
this._container = null;
},
/**
* Creates an empty, hidden iframe and sets it to the _iframe
* property of this object.
*
* @return frame
* (iframe) An empty, hidden iframe
*/
_createFrame: function _createFrame() {
let doc = Services.wm.getMostRecentWindow("navigator:browser").document;
// Insert iframe in to create docshell.
let frame = doc.createElement("iframe");
frame.setAttribute("type", "content");
frame.setAttribute("collapsed", "true");
doc.documentElement.appendChild(frame);
// Stop about:blank from being loaded.
let webNav = frame.docShell.QueryInterface(Ci.nsIWebNavigation);
webNav.stop(Ci.nsIWebNavigation.STOP_NETWORK);
// Set instance properties.
this._frame = frame;
this._container = doc.documentElement;
},
_createSandbox: function _createSandbox(cb, uri) {
let self = this;
this._frame.addEventListener(
"DOMContentLoaded",
function _makeSandboxContentLoaded(event) {
if (event.target.location.toString() != uri) {
return;
}
event.target.removeEventListener(
"DOMContentLoaded", _makeSandboxContentLoaded, false
);
let workerWindow = self._frame.contentWindow;
self.box = new Cu.Sandbox(workerWindow, {
wantXrays: false,
sandboxPrototype: workerWindow
});
cb(self);
},
true
);
// Load the iframe.
this._frame.docShell.loadURI(
uri,
this._frame.docShell.LOAD_FLAGS_NONE,
null, // referrer
null, // postData
null // headers
);
},
};
XPCOMUtils.defineLazyGetter(this, "BrowserID", function() {
return new BrowserIDService();
});

387
services/aitc/modules/client.js Normal file
View File

@ -0,0 +1,387 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this file,
* You can obtain one at http://mozilla.org/MPL/2.0/. */
"use strict";
const EXPORTED_SYMBOLS = ["AitcClient"];
const {classes: Cc, interfaces: Ci, utils: Cu, results: Cr} = Components;
Cu.import("resource://gre/modules/Webapps.jsm");
Cu.import("resource://services-common/log4moz.js");
Cu.import("resource://services-common/preferences.js");
Cu.import("resource://services-common/rest.js");
Cu.import("resource://services-common/utils.js");
Cu.import("resource://services-crypto/utils.js");
/**
* First argument is a token as returned by CommonUtils.TokenServerClient.
* This is a convenience, so you don't have to call updateToken immediately
* after making a new client (though you must when the token expires and you
* intend to reuse the same client instance).
*
* Second argument is a key-value store object that exposes two methods:
* set(key, value); Sets the value of a given key
* get(key, default); Returns the value of key, if it doesn't exist,
* return default
* The values should be stored persistently. The Preferences object from
* services-common/preferences.js is an example of such an object.
*/
function AitcClient(token, state) {
this.updateToken(token);
this._log = Log4Moz.repository.getLogger("Service.AITC.Client");
this._log.level = Log4Moz.Level[
Preferences.get("services.aitc.client.log.level")
];
this._state = state;
this._backoff = !!state.get("backoff", false);
this._timeout = state.get("timeout", 120);
this._appsLastModified = parseInt(state.get("lastModified", "0"), 10);
this._log.info("Client initialized with token endpoint: " + this.uri);
}
AitcClient.prototype = {
_requiredLocalKeys: [
"origin", "receipts", "manifestURL", "installOrigin"
],
_requiredRemoteKeys: [
"origin", "name", "receipts", "manifestPath", "installOrigin",
"installedAt", "modifiedAt"
],
/**
* Updates the token the client must use to authenticate outgoing requests.
*
* @param token
* (Object) A token as returned by CommonUtils.TokenServerClient.
*/
updateToken: function updateToken(token) {
this.uri = token.endpoint.replace(/\/+$/, "");
this.token = {id: token.id, key: token.key};
},
/**
* Initiates an update of a newly installed app to the AITC server. Call this
* when an application is installed locally.
*
* @param app
* (Object) The app record of the application that was just installed.
*/
remoteInstall: function remoteInstall(app, cb) {
if (!cb) {
throw new Error("remoteInstall called without callback");
}
// Fetch the name of the app because it's not in the event app object.
let self = this;
DOMApplicationRegistry.getManifestFor(app.origin, function gotManifest(m) {
app.name = m.name;
self._putApp(self._makeRemoteApp(app), cb);
});
},
/**
* Initiates an update of an uinstalled app to the AITC server. Call this
* when an application is uninstalled locally.
*
* @param app
* (Object) The app record of the application that was uninstalled.
*/
remoteUninstall: function remoteUninstall(app, cb) {
if (!cb) {
throw new Error("remoteUninstall called without callback");
}
app.name = "Uninstalled"; // Bug 760262
let record = this._makeRemoteApp(app);
record.deleted = true;
this._putApp(record, cb);
},
/**
* Fetch remote apps from server with GET. The provided callback will receive
* an array of app objects in the format expected by DOMApplicationRegistry,
* if successful, or an Error; in the usual (err, result) way.
*/
getApps: function getApps(cb) {
if (!cb) {
throw new Error("getApps called but no callback provided");
}
if (!this._isRequestAllowed()) {
cb(null, null);
return;
}
let uri = this.uri + "/apps/?full=1";
let req = new TokenAuthenticatedRESTRequest(uri, this.token);
req.timeout = this._timeout;
req.setHeader("Content-Type", "application/json");
if (this._appsLastModified) {
req.setHeader("X-If-Modified-Since", this._appsLastModified);
}
let self = this;
req.get(function _getAppsCb(err) {
self._processGetApps(err, cb, req);
});
},
/**
* GET request returned from getApps, process.
*/
_processGetApps: function _processGetApps(err, cb, req) {
// Set X-Backoff or Retry-After, if needed.
this._setBackoff(req);
if (err) {
this._log.error("getApps request error " + err);
cb(err, null);
return;
}
// Bubble auth failure back up so new token can be acquired.
if (req.response.status == 401) {
let msg = new Error("getApps failed due to 401 authentication failure");
this._log.info(msg);
msg.authfailure = true;
cb(msg, null);
return;
}
// Process response.
if (req.response.status == 304) {
this._log.info("getApps returned 304");
cb(null, null);
return;
}
if (req.response.status != 200) {
this._log.error(req);
cb(new Error("Unexpected error with getApps"), null);
return;
}
let apps;
try {
let tmp = JSON.parse(req.response.body);
tmp = tmp["apps"];
// Convert apps from remote to local format.
apps = tmp.map(this._makeLocalApp, this);
this._log.info("getApps succeeded and got " + apps.length);
} catch (e) {
this._log.error(CommonUtils.exceptionStr(e));
cb(new Error("Exception in getApps " + e), null);
return;
}
// Return success.
try {
cb(null, apps);
// Don't update lastModified until we know cb succeeded.
this._appsLastModified = parseInt(req.response.headers["X-Timestamp"], 10);
this._state.set("lastModified", "" + this._appsLastModified);
} catch (e) {
this._log.error("Exception in getApps callback " + e);
}
},
/**
* Change a given app record to match what the server expects.
* Change manifestURL to manifestPath, and trim out manifests since we
* don't store them on the server.
*/
_makeRemoteApp: function _makeRemoteApp(app) {
for each (let key in this.requiredLocalKeys) {
if (!(key in app)) {
throw new Error("Local app missing key " + key);
}
}
let record = {
name: app.name,
origin: app.origin,
receipts: app.receipts,
manifestPath: app.manifestURL,
installOrigin: app.installOrigin
};
if ("modifiedAt" in app) {
record.modifiedAt = app.modifiedAt;
}
if ("installedAt" in app) {
record.installedAt = app.installedAt;
}
return record;
},
/**
* Change a given app record received from the server to match what the local
* registry expects. (Inverse of _makeRemoteApp)
*/
_makeLocalApp: function _makeLocalApp(app) {
for each (let key in this._requiredRemoteKeys) {
if (!(key in app)) {
throw new Error("Remote app missing key " + key);
}
}
let record = {
origin: app.origin,
installOrigin: app.installOrigin,
installedAt: app.installedAt,
modifiedAt: app.modifiedAt,
manifestURL: app.manifestPath,
receipts: app.receipts
};
if ("deleted" in app) {
record.deleted = app.deleted;
}
return record;
},
/**
* Try PUT for an app on the server and determine if we should retry
* if it fails.
*/
_putApp: function _putApp(app, cb) {
if (!this._isRequestAllowed()) {
// PUT requests may qualify as the "minimum number of additional requests
// required to maintain consistency of their stored data". However, it's
// better to keep server load low, even if it means user's apps won't
// reach their other devices during the early days of AITC. We should
// revisit this when we have a better of idea of server load curves.
err = new Error("Backoff in effect, aborting PUT");
err.processed = false;
cb(err, null);
return;
}
let uri = this._makeAppURI(app.origin);
let req = new TokenAuthenticatedRESTRequest(uri, this.token);
req.timeout = this._timeout;
req.setHeader("Content-Type", "application/json");
if (app.modifiedAt) {
req.setHeader("X-If-Unmodified-Since", "" + app.modified);
}
let self = this;
this._log.info("Trying to _putApp to " + uri);
req.put(JSON.stringify(app), function _putAppCb(err) {
self._processPutApp(err, cb, req);
});
},
/**
* PUT from _putApp finished, process.
*/
_processPutApp: function _processPutApp(error, cb, req) {
this._setBackoff(req);
if (error) {
this._log.error("_putApp request error " + error);
cb(error, null);
return;
}
let err = null;
switch (req.response.status) {
case 201:
case 204:
this._log.info("_putApp succeeded");
cb(null, true);
break;
case 401:
// Bubble auth failure back up so new token can be acquired
err = new Error("_putApp failed due to 401 authentication failure");
this._log.warn(err);
err.authfailure = true;
cb(err, null);
break;
case 409:
// Retry on server conflict
err = new Error("_putApp failed due to 409 conflict");
this._log.warn(err);
cb(err,null);
break;
case 400:
case 412:
case 413:
let msg = "_putApp returned: " + req.response.status;
this._log.warn(msg);
err = new Error(msg);
err.processed = true;
cb(err, null);
break;
default:
this._error(req);
err = new Error("Unexpected error with _putApp");
err.processed = false;
cb(err, null);
break;
}
},
/**
* Utility methods.
*/
_error: function _error(req) {
this._log.error("Catch-all error for request: " +
req.uri.asciiSpec + req.response.status + " with: " + req.response.body);
},
_makeAppURI: function _makeAppURI(origin) {
let part = CommonUtils.encodeBase64URL(
CryptoUtils.UTF8AndSHA1(origin)
).replace("=", "");
return this.uri + "/apps/" + part;
},
// Before making a request, check if we are allowed to.
_isRequestAllowed: function _isRequestAllowed() {
if (!this._backoff) {
return true;
}
let time = Date.now();
let backoff = parseInt(this._state.get("backoff", 0), 10);
if (time < backoff) {
this._log.warn(backoff - time + "ms left for backoff, aborting request");
return false;
}
this._backoff = false;
this._state.set("backoff", "0");
return true;
},
// Set values from X-Backoff and Retry-After headers, if present
_setBackoff: function _setBackoff(req) {
let backoff = 0;
let val;
if (req.response.headers["Retry-After"]) {
val = req.response.headers["Retry-After"];
backoff = parseInt(val, 10);
this._log.warn("Retry-Header header was seen: " + val);
} else if (req.response.headers["X-Backoff"]) {
val = req.response.headers["X-Backoff"];
backoff = parseInt(val, 10);
this._log.warn("X-Backoff header was seen: " + val);
}
if (backoff) {
this._backoff = true;
let time = Date.now();
// Fuzz backoff time so all client don't retry at the same time
backoff = Math.floor((Math.random() * backoff + backoff) * 1000);
this._state.set("backoff", "" + (time + backoff));
}
},
};

161
services/aitc/modules/main.js Normal file
View File

@ -0,0 +1,161 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this file,
* You can obtain one at http://mozilla.org/MPL/2.0/. */
"use strict";
const EXPORTED_SYMBOLS = ["Aitc"];
const {classes: Cc, interfaces: Ci, utils: Cu} = Components;
Cu.import("resource://gre/modules/Webapps.jsm");
Cu.import("resource://gre/modules/Services.jsm");
Cu.import("resource://gre/modules/XPCOMUtils.jsm");
Cu.import("resource://services-aitc/manager.js");
Cu.import("resource://services-common/utils.js");
Cu.import("resource://services-common/log4moz.js");
Cu.import("resource://services-common/preferences.js");
function Aitc() {
this._log = Log4Moz.repository.getLogger("Service.AITC");
this._log.level = Log4Moz.Level[Preferences.get(
"services.aitc.service.log.level"
)];
this._log.info("Loading AitC");
this.DASHBOARD_ORIGIN = CommonUtils.makeURI(
Preferences.get("services.aitc.dashboard.url")
).prePath;
this._manager = new AitcManager(this._init.bind(this));
}
Aitc.prototype = {
// The goal of the init function is to be ready to activate the AITC
// client whenever the user is looking at the dashboard.
_init: function init() {
let self = this;
// This is called iff the user is currently looking the dashboard.
function dashboardLoaded(browser) {
let win = browser.contentWindow;
self._log.info("Dashboard was accessed " + win);
// If page is ready to go, fire immediately.
if (win.document && win.document.readyState == "complete") {
self._manager.userActive(win);
return;
}
// Only fire event after the page fully loads.
browser.contentWindow.addEventListener(
"DOMContentLoaded",
function _contentLoaded(event) {
self._manager.userActive(win);
},
false
);
}
// This is called when the user's attention is elsewhere.
function dashboardUnloaded() {
self._log.info("Dashboard closed or in background");
self._manager.userIdle();
}
// Called when a URI is loaded in any tab. We have to listen for this
// because tabSelected is not called if I open a new tab which loads
// about:home and then navigate to the dashboard, or navigation via
// links on the currently open tab.
let listener = {
onLocationChange: function onLocationChange(browser, pr, req, loc, flag) {
let win = Services.wm.getMostRecentWindow("navigator:browser");
if (win.gBrowser.selectedBrowser == browser) {
if (loc.prePath == self.DASHBOARD_ORIGIN) {
dashboardLoaded(browser);
}
}
}
};
// Called when the current tab selection changes.
function tabSelected(event) {
let browser = event.target.linkedBrowser;
if (browser.currentURI.prePath == self.DASHBOARD_ORIGIN) {
dashboardLoaded(browser);
} else {
dashboardUnloaded();
}
}
// Add listeners for all windows opened in the future.
function winWatcher(subject, topic) {
if (topic != "domwindowopened") {
return;
}
subject.addEventListener("load", function winWatcherLoad() {
subject.removeEventListener("load", winWatcherLoad, false);
let doc = subject.document.documentElement;
if (doc.getAttribute("windowtype") == "navigator:browser") {
let browser = subject.gBrowser;
browser.addTabsProgressListener(listener);
browser.tabContainer.addEventListener("TabSelect", tabSelected);
}
}, false);
}
Services.ww.registerNotification(winWatcher);
// Add listeners for all current open windows.
let enumerator = Services.wm.getEnumerator("navigator:browser");
while (enumerator.hasMoreElements()) {
let browser = enumerator.getNext().gBrowser;
browser.addTabsProgressListener(listener);
browser.tabContainer.addEventListener("TabSelect", tabSelected);
// Also check the currently open URI.
if (browser.currentURI.prePath == this.DASHBOARD_ORIGIN) {
dashboardLoaded(browser);
}
}
// Add listeners for app installs/uninstall.
Services.obs.addObserver(this, "webapps-sync-install", false);
Services.obs.addObserver(this, "webapps-sync-uninstall", false);
// Add listener for idle service.
let idleSvc = Cc["@mozilla.org/widget/idleservice;1"].
getService(Ci.nsIIdleService);
idleSvc.addIdleObserver(this,
Preferences.get("services.aitc.main.idleTime"));
},
observe: function(subject, topic, data) {
let app;
switch (topic) {
case "webapps-sync-install":
app = JSON.parse(data);
this._log.info(app.origin + " was installed, initiating PUT");
this._manager.appEvent("install", app);
break;
case "webapps-sync-uninstall":
app = JSON.parse(data);
this._log.info(app.origin + " was uninstalled, initiating PUT");
this._manager.appEvent("uninstall", app);
break;
case "idle":
this._log.info("User went idle");
if (this._manager) {
this._manager.userIdle();
}
break;
case "back":
this._log.info("User is no longer idle");
let win = Services.wm.getMostRecentWindow("navigator:browser");
if (win && win.gBrowser.currentURI.prePath == this.DASHBOARD_ORIGIN &&
this._manager) {
this._manager.userActive();
}
break;
}
},
};

573
services/aitc/modules/manager.js Normal file
View File

@ -0,0 +1,573 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this file,
* You can obtain one at http://mozilla.org/MPL/2.0/. */
"use strict";
const EXPORTED_SYMBOLS = ["AitcManager"];
const {classes: Cc, interfaces: Ci, utils: Cu} = Components;
Cu.import("resource://gre/modules/Webapps.jsm");
Cu.import("resource://gre/modules/Services.jsm");
Cu.import("resource://gre/modules/FileUtils.jsm");
Cu.import("resource://services-aitc/client.js");
Cu.import("resource://services-aitc/browserid.js");
Cu.import("resource://services-aitc/storage.js");
Cu.import("resource://services-common/log4moz.js");
Cu.import("resource://services-common/preferences.js");
Cu.import("resource://services-common/tokenserverclient.js");
Cu.import("resource://services-common/utils.js");
const PREFS = new Preferences("services.aitc.");
const TOKEN_TIMEOUT = 240000; // 4 minutes
const DASHBOARD_URL = PREFS.get("dashboard.url");
const MARKETPLACE_URL = PREFS.get("marketplace.url");
/**
* The constructor for the manager takes a callback, which will be invoked when
* the manager is ready (construction is asynchronous). *DO NOT* call any
* methods on this object until the callback has been invoked, doing so will
* lead to undefined behaviour.
*/
function AitcManager(cb) {
this._client = null;
this._getTimer = null;
this._putTimer = null;
this._lastToken = 0;
this._lastEmail = null;
this._dashboardWindow = null;
this._log = Log4Moz.repository.getLogger("Service.AITC.Manager");
this._log.level = Log4Moz.Level[Preferences.get("manager.log.level")];
this._log.info("Loading AitC manager module");
// Check if we have pending PUTs from last time.
let self = this;
this._pending = new AitcQueue("webapps-pending.json", function _queueDone() {
// Inform the AitC service that we're good to go!
self._log.info("AitC manager has finished loading");
try {
cb(true);
} catch (e) {
self._log.error(new Error("AitC manager callback threw " + e));
}
// Schedule them, but only if we can get a silent assertion.
self._makeClient(function(err, client) {
if (!err && client) {
self._client = client;
self._processQueue();
}
}, false);
});
}
AitcManager.prototype = {
/**
* State of the user. ACTIVE implies user is looking at the dashboard,
* PASSIVE means either not at the dashboard or the idle timer started.
*/
_ACTIVE: 1,
_PASSIVE: 2,
/**
* Smart setter that will only call _setPoll is the value changes.
*/
_clientState: null,
get _state() {
return this._clientState;
},
set _state(value) {
if (this._clientState == value) {
return;
}
this._clientState = value;
this._setPoll();
},
/**
* Local app was just installed or uninstalled, ask client to PUT if user
* is logged in.
*/
appEvent: function appEvent(type, app) {
// Add this to the equeue.
let self = this;
let obj = {type: type, app: app, retries: 0, lastTime: 0};
this._pending.enqueue(obj, function _enqueued(err, rec) {
if (err) {
self._log.error("Could not add " + type + " " + app + " to queue");
return;
}
// If we already have a client (i.e. user is logged in), attempt to PUT.
if (self._client) {
self._processQueue();
return;
}
// If not, try a silent client creation.
self._makeClient(function(err, client) {
if (!err && client) {
self._client = client;
self._processQueue();
}
// If user is not logged in, we'll just have to try later.
});
});
},
/**
* User is looking at dashboard. Start polling actively, but if user isn't
* logged in, prompt for them to login via a dialog.
*/
userActive: function userActive(win) {
// Stash a reference to the dashboard window in case we need to prompt
this._dashboardWindow = win;
if (this._client) {
this._state = this._ACTIVE;
return;
}
// Make client will first try silent login, if it doesn't work, a popup
// will be shown in the context of the dashboard. We shouldn't be
// trying to make a client every time this function is called, there is
// room for optimization (Bug 750607).
let self = this;
this._makeClient(function(err, client) {
if (err) {
// Notify user of error (Bug 750610).
self._log.error("Client not created at Dashboard");
return;
}
self._client = client;
self._state = self._ACTIVE;
}, true, win);
},
/**
* User is idle, (either by idle observer, or by not being on the dashboard).
* When the user is no longer idle and the dashboard is the current active
* page, a call to userActive MUST be made.
*/
userIdle: function userIdle() {
this._state = this._PASSIVE;
this._dashboardWindow = null;
},
/**
* Poll the AITC server for any changes and process them. It is safe to call
* this function multiple times. Last caller wins. The function will
* grab the current user state from _state and act accordingly.
*
* Invalid states will cause this function to throw.
*/
_setPoll: function _setPoll() {
if (this._state == this._ACTIVE && !this._client) {
throw new Error("_setPoll(ACTIVE) called without client");
}
if (this._state != this._ACTIVE && this._state != this._PASSIVE) {
throw new Error("_state is invalid " + this._state);
}
if (!this._client) {
// User is not logged in, we cannot do anything.
self._log.warn("_setPoll called but user not logged in, ignoring");
return;
}
// Check if there are any PUTs pending first.
if (this._pending.length && !(this._putTimer)) {
// There are pending PUTs and no timer, so let's process them. GETs will
// resume after the PUTs finish (see processQueue)
this._processQueue();
return;
}
// Do one GET soon, but only if user is active.
let getFreq;
if (this._state == this._ACTIVE) {
CommonUtils.nextTick(this._checkServer, this);
getFreq = PREFS.get("manager.getActiveFreq");
} else {
getFreq = PREFS.get("manager.getPassiveFreq");
}
// Cancel existing timer, if any.
if (this._getTimer) {
this._getTimer.cancel();
this._getTimer = null;
}
// Start the timer for GETs.
let self = this;
this._log.info("Starting GET timer");
this._getTimer = Cc["@mozilla.org/timer;1"].createInstance(Ci.nsITimer);
this._getTimer.initWithCallback({notify: this._checkServer.bind(this)},
getFreq, Ci.nsITimer.TYPE_REPEATING_SLACK);
this._log.info("GET timer set, next attempt in " + getFreq + "ms");
},
/**
* Checks if the current token we hold is valid. If not, we obtain a new one
* and execute the provided func. If a token could not be obtained, func will
* not be called and an error will be logged.
*/
_validateToken: function _validateToken(func) {
if (Date.now() - this._lastToken < TOKEN_TIMEOUT) {
func();
return;
}
let win;
if (this._state == this.ACTIVE) {
win = this._dashboardWindow;
}
let self = this;
this._refreshToken(function(err, done) {
if (!done) {
this._log.warn("_checkServer could not refresh token, aborting");
return;
}
func();
}, win);
},
/**
* Do a GET check on the server to see if we have any new apps. Abort if
* there are pending PUTs. If we GET some apps, send to storage for
* further processing.
*/
_checkServer: function _checkServer() {
if (!this._client) {
throw new Error("_checkServer called without a client");
}
if (this._pending.length) {
this._log.warn("_checkServer aborted because of pending PUTs");
return;
}
this._validateToken(this._getApps.bind(this));
},
_getApps: function _getApps() {
// Do a GET
this._log.info("Attempting to getApps");
let self = this;
this._client.getApps(function gotApps(err, apps) {
if (err) {
// Error was logged in client.
return;
}
if (!apps) {
// No changes, got 304.
return;
}
if (!apps.length) {
// Empty array, nothing to process
self._log.info("No apps found on remote server");
return;
}
// Send list of remote apps to storage to apply locally
AitcStorage.processApps(apps, function processedApps() {
self._log.info("processApps completed successfully, changes applied");
});
});
},
/**
* Go through list of apps to PUT and attempt each one. If we fail, try
* again in PUT_FREQ. Will throw if called with an empty, _reschedule()
* makes sure that we don't.
*/
_processQueue: function _processQueue() {
if (!this._client) {
throw new Error("_processQueue called without a client");
}
if (!this._pending.length) {
throw new Error("_processQueue called with an empty queue");
}
if (this._putInProgress) {
// The network request sent out as a result to the last call to
// _processQueue still isn't done. A timer is created they all
// finish to make sure this function is called again if neccessary.
return;
}
this._validateToken(this._putApps.bind(this));
},
_putApps: function _putApps() {
this._putInProgress = true;
let record = this._pending.peek();
this._log.info("Processing record type " + record.type);
let self = this;
function _clientCallback(err, done) {
// Send to end of queue if unsuccessful or err.removeFromQueue is false.
if (err && !err.removeFromQueue) {
self._log.info("PUT failed, re-adding to queue");
// Update retries and time
record.retries += 1;
record.lastTime = new Date().getTime();
// Add updated record to the end of the queue.
self._pending.enqueue(record, function(err, done) {
if (err) {
self._log.error("Enqueue failed " + err);
_reschedule();
return;
}
// If record was successfully added, remove old record.
self._pending.dequeue(function(err, done) {
if (err) {
self._log.error("Dequeue failed " + err);
}
_reschedule();
return;
});
});
}
// If succeeded or client told us to remove from queue
self._log.info("_putApp asked us to remove it from queue");
self._pending.dequeue(function(err, done) {
if (err) {
self._log.error("Dequeue failed " + e);
}
_reschedule();
});
}
function _reschedule() {
// Release PUT lock
self._putInProgress = false;
// We just finished PUTting an object, try the next one immediately,
// but only if haven't tried it already in the last putFreq (ms).
if (!self._pending.length) {
// Start GET timer now that we're done with PUTs.
self._setPoll();
return;
}
let obj = self._pending.peek();
let cTime = new Date().getTime();
let freq = PREFS.get("manager.putFreq");
// We tried this object recently, we'll come back to it later.
if (obj.lastTime && ((cTime - obj.lastTime) < freq)) {
self._log.info("Scheduling next processQueue in " + freq);
CommonUtils.namedTimer(self._processQueue, freq, self, "_putTimer");
return;
}
// Haven't tried this PUT yet, do it immediately.
self._log.info("Queue non-empty, processing next PUT");
self._processQueue();
}
switch (record.type) {
case "install":
this._client.remoteInstall(record.app, _clientCallback);
break;
case "uninstall":
record.app.deleted = true;
this._client.remoteUninstall(record.app, _clientCallback);
break;
default:
this._log.warn(
"Unrecognized type " + record.type + " in queue, removing"
);
let self = this;
this._pending.dequeue(function _dequeued(err) {
if (err) {
self._log.error("Dequeue of unrecognized app type failed");
}
_reschedule();
});
}
},
/* Obtain a (new) token from the Sagrada token server. If win is is specified,
* the user will be asked to login via UI, if required. The callback's
* signature is cb(err, done). If a token is obtained successfully, done will
* be true and err will be null.
*/
_refreshToken: function _refreshToken(cb, win) {
if (!this._client) {
throw new Error("_refreshToken called without an active client");
}
this._log.info("Token refresh requested");
let self = this;
function refreshedAssertion(err, assertion) {
if (!err) {
self._getToken(assertion, function(err, token) {
if (err) {
cb(err, null);
return;
}
self._lastToken = Date.now();
self._client.updateToken(token);
cb(null, true);
});
return;
}
// Silent refresh was asked for.
if (!win) {
cb(err, null);
return;
}
// Prompt user to login.
self._makeClient(function(err, client) {
if (err) {
cb(err, null);
return;
}
// makeClient sets an updated token.
self._client = client;
cb(null, true);
}, win);
}
let options = { audience: DASHBOARD_URL };
if (this._lastEmail) {
options.requiredEmail = this._lastEmail;
} else {
options.sameEmailAs = MARKETPLACE_URL;
}
BrowserID.getAssertion(refreshedAssertion, options);
},
/* Obtain a token from Sagrada token server, given a BrowserID assertion
* cb(err, token) will be invoked on success or failure.
*/
_getToken: function _getToken(assertion, cb) {
let url = PREFS.get("tokenServer.url") + "/1.0/aitc/1.0";
let client = new TokenServerClient();
this._log.info("Obtaining token from " + url);
let self = this;
try {
client.getTokenFromBrowserIDAssertion(url, assertion, function(err, tok) {
self._gotToken(err, tok, cb);
});
} catch (e) {
cb(new Error(e), null);
}
},
// Token recieved from _getToken.
_gotToken: function _gotToken(err, tok, cb) {
if (!err) {
this._log.info("Got token from server: " + JSON.stringify(tok));
cb(null, tok);
return;
}
let msg = err.name + " in _getToken: " + err.error;
this._log.error(msg);
cb(msg, null);
},
// Extract the email address from a BrowserID assertion.
_extractEmail: function _extractEmail(assertion) {
// Please look the other way while I do this. Thanks.
let chain = assertion.split("~");
let len = chain.length;
if (len < 2) {
return null;
}
try {
// We need CommonUtils.decodeBase64URL.
let cert = JSON.parse(atob(
chain[0].split(".")[1].replace("-", "+", "g").replace("_", "/", "g")
));
return cert.principal.email;
} catch (e) {
return null;
}
},
/* To start the AitcClient we need a token, for which we need a BrowserID
* assertion. If login is true, makeClient will ask the user to login in
* the context of win. cb is called with (err, client).
*/
_makeClient: function makeClient(cb, login, win) {
if (!cb) {
throw new Error("makeClient called without callback");
}
if (login && !win) {
throw new Error("makeClient called with login as true but no win");
}
let self = this;
let ctxWin = win;
function processAssertion(val) {
// Store the email we got the token for so we can refresh.
self._lastEmail = self._extractEmail(val);
self._log.info("Got assertion from BrowserID, creating token");
self._getToken(val, function(err, token) {
if (err) {
cb(err, null);
return;
}
// Store when we got the token so we can refresh it as needed.
self._lastToken = Date.now();
// We only create one client instance, store values in a pref tree
cb(null, new AitcClient(
token, new Preferences("services.aitc.client.")
));
});
}
function gotSilentAssertion(err, val) {
self._log.info("gotSilentAssertion called");
if (err) {
// If we were asked to let the user login, do the popup method.
if (login) {
self._log.info("Could not obtain silent assertion, retrying login");
BrowserID.getAssertionWithLogin(function gotAssertion(err, val) {
if (err) {
self._log.error(err);
cb(err, false);
return;
}
processAssertion(val);
}, ctxWin);
return;
}
self._log.warn("Could not obtain assertion in _makeClient");
cb(err, false);
} else {
processAssertion(val);
}
}
// Check if we can get assertion silently first
self._log.info("Attempting to obtain assertion silently")
BrowserID.getAssertion(gotSilentAssertion, {
audience: DASHBOARD_URL,
sameEmailAs: MARKETPLACE_URL
});
},
};

452
services/aitc/modules/storage.js Normal file
View File

@ -0,0 +1,452 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this file,
* You can obtain one at http://mozilla.org/MPL/2.0/. */
"use strict";
const EXPORTED_SYMBOLS = ["AitcStorage", "AitcQueue"];
const {classes: Cc, interfaces: Ci, utils: Cu, results: Cr} = Components;
Cu.import("resource://gre/modules/NetUtil.jsm");
Cu.import("resource://gre/modules/Webapps.jsm");
Cu.import("resource://gre/modules/FileUtils.jsm");
Cu.import("resource://gre/modules/XPCOMUtils.jsm");
Cu.import("resource://services-common/log4moz.js");
Cu.import("resource://services-common/preferences.js");
Cu.import("resource://services-common/rest.js");
/**
* Provides a file-backed queue. Currently used by manager.js as persistent
* storage to manage pending installs and uninstalls.
*
* @param filename
* (String) The file backing this queue will be named as this string.
*
* @param cb
* (Function) This function will be called when the queue is ready to
* use. *DO NOT* call any methods on this object until the
* callback is invoked, if you do so, none of your operations
* will be persisted on disk.
*
*/
function AitcQueue(filename, cb) {
if (!cb) {
throw new Error("AitcQueue constructor called without callback");
}
this._log = Log4Moz.repository.getLogger("Service.AITC.Storage.Queue");
this._log.level = Log4Moz.Level[Preferences.get(
"services.aitc.storage.log.level"
)];
this._queue = [];
this._writeLock = false;
this._file = FileUtils.getFile("ProfD", ["webapps", filename], true);
this._log.info("AitcQueue instance loading");
let self = this;
if (this._file.exists()) {
this._getFile(function gotFile(data) {
if (data && Array.isArray(data)) {
self._queue = data;
}
self._log.info("AitcQueue instance created");
cb(true);
});
} else {
self._log.info("AitcQueue instance created");
cb(true);
}
}
AitcQueue.prototype = {
/**
* Add an object to the queue, and data is saved to disk.
*/
enqueue: function enqueue(obj, cb) {
this._log.info("Adding to queue " + obj);
if (!cb) {
throw new Error("enqueue called without callback");
}
let self = this;
this._queue.push(obj);
try {
this._putFile(this._queue, function _enqueuePutFile(err) {
if (err) {
// Write unsuccessful, don't add to queue.
self._queue.pop();
cb(err, false);
return;
}
// Successful write.
cb(null, true);
return;
});
} catch (e) {
self._queue.pop();
cb(e, false);
}
},
/**
* Remove the object at the head of the queue, and data is saved to disk.
*/
dequeue: function dequeue(cb) {
this._log.info("Removing head of queue");
if (!cb) {
throw new Error("dequeue called without callback");
}
if (!this._queue.length) {
throw new Error("Queue is empty");
}
let self = this;
let obj = this._queue.shift();
try {
this._putFile(this._queue, function _dequeuePutFile(err) {
if (!err) {
// Successful write.
cb(null, true);
return;
}
// Unsuccessful write, put back in queue.
self._queue.unshift(obj);
cb(err, false);
});
} catch (e) {
self._queue.unshift(obj);
cb(e, false);
}
},
/**
* Return the object at the front of the queue without removing it.
*/
peek: function peek() {
this._log.info("Peek called when length " + this._queue.length);
if (!this._queue.length) {
throw new Error("Queue is empty");
}
return this._queue[0];
},
/**
* Find out the length of the queue.
*/
get length() {
return this._queue.length;
},
/**
* Get contents of cache file and parse it into an array. Will throw an
* exception if there is an error while reading the file.
*/
_getFile: function _getFile(cb) {
let channel = NetUtil.newChannel(this._file);
channel.contentType = "application/json";
let self = this;
NetUtil.asyncFetch(channel, function _asyncFetched(stream, res) {
if (!Components.isSuccessCode(res)) {
self._log.error("Could not read from json file " + this._file.path);
cb(null);
return;
}
let data = [];
try {
data = JSON.parse(
NetUtil.readInputStreamToString(stream, stream.available())
);
stream.close();
cb(data);
} catch (e) {
self._log.error("Could not parse JSON " + e);
cb(null);
}
});
},
/**
* Put an array into the cache file. Will throw an exception if there is
* an error while trying to write to the file.
*/
_putFile: function _putFile(value, cb) {
if (this._writeLock) {
throw new Error("_putFile already in progress");
}
this._writeLock = true;
try {
let ostream = FileUtils.openSafeFileOutputStream(this._file);
let converter = Cc["@mozilla.org/intl/scriptableunicodeconverter"].
createInstance(Ci.nsIScriptableUnicodeConverter);
converter.charset = "UTF-8";
let istream = converter.convertToInputStream(JSON.stringify(value));
// Asynchronously copy the data to the file.
let self = this;
this._log.info("Writing queue to disk");
NetUtil.asyncCopy(istream, ostream, function _asyncCopied(result) {
self._writeLock = false;
if (Components.isSuccessCode(result)) {
self._log.info("asyncCopy succeeded");
cb(null);
} else {
let msg = new Error("asyncCopy failed with " + result);
self._log.info(msg);
cb(msg);
}
});
} catch (e) {
this._writeLock = false;
cb(msg);
}
},
};
/**
* An interface to DOMApplicationRegistry, used by manager.js to process
* remote changes received and apply them to the local registry.
*/
function AitcStorageImpl() {
this._log = Log4Moz.repository.getLogger("Service.AITC.Storage");
this._log.level = Log4Moz.Level[Preferences.get(
"services.aitc.storage.log.level"
)];
this._log.info("Loading AitC storage module");
}
AitcStorageImpl.prototype = {
/**
* Determines what changes are to be made locally, given a list of
* remote apps.
*
* @param remoteApps
* (Array) An array of app records fetched from the AITC server.
*
* @param callback
* (function) A callback to be invoked when processing is finished.
*/
processApps: function processApps(remoteApps, callback) {
let self = this;
this._log.info("Server check got " + remoteApps.length + " apps");
// Get the set of local apps, and then pass to _processApps.
// _processApps will check for the validity of remoteApps.
DOMApplicationRegistry.getAllWithoutManifests(
function _processAppsGotLocalApps(localApps) {
self._processApps(remoteApps, localApps, callback);
}
);
},
/**
* Take a list of remote and local apps and figured out what changes (if any)
* are to be made to the local DOMApplicationRegistry.
*
* General algorithm:
* 1. Put all remote apps in a dictionary of origin->app.
* 2. Put all local apps in a dictionary of origin->app.
* 3. Mark all local apps as "to be deleted".
* 4. Go through each remote app:
* 4a. If remote app is not marked as deleted, remove from the "to be
* deleted" set.
* 4b. If remote app is marked as deleted, but isn't present locally,
* process the next remote app.
* 4c. If remote app is not marked as deleted and isn't present locally,
* add to the "to be installed" set.
* 5. For each app either in the "to be installed" or "to be deleted" set,
* apply the changes locally. For apps to be installed, we must also
* fetch the manifest.
*
*/
_processApps: function _processApps(remoteApps, lApps, callback) {
let toDelete = {};
let localApps = {};
// If remoteApps is empty, do nothing. The correct thing to do is to
// delete all local apps, but we'll play it safe for now since we are
// marking apps as deleted anyway. In a subsequent version (when the
// deleted flag is no longer in use), this check can be removed.
if (!Object.keys(remoteApps).length) {
this._log.warn("Empty set of remote apps to _processApps, returning");
callback();
return;
}
// Convert lApps to a dictionary of origin -> app (instead of id -> app).
for (let [id, app] in Iterator(lApps)) {
app.id = id;
toDelete[app.origin] = app;
localApps[app.origin] = app;
}
// Iterate over remote apps, and find out what changes we must apply.
let toInstall = [];
for each (let app in remoteApps) {
// Don't delete apps that are both local & remote.
let origin = app.origin;
if (!app.deleted) {
delete toDelete[origin];
}
// A remote app that was deleted, but also isn't present locally is NOP.
if (app.deleted && !localApps[origin]) {
continue;
}
// If there is a remote app that isn't local or if the remote app was
// installed or updated later.
let id;
if (!localApps[origin]) {
id = DOMApplicationRegistry.makeAppId();
}
if (localApps[origin] &&
(localApps[origin].installTime < app.installTime)) {
id = localApps[origin].id;
}
// We should (re)install this app locally
if (id) {
toInstall.push({id: id, value: app});
}
}
// Uninstalls only need the ID & deleted flag.
let toUninstall = [];
for (let origin in toDelete) {
toUninstall.push({id: toDelete[origin].id, deleted: true});
}
// Apply uninstalls first, we do not need to fetch manifests.
if (toUninstall.length) {
this._log.info("Applying uninstalls to registry");
let self = this;
DOMApplicationRegistry.updateApps(toUninstall, function() {
// If there are installs, proceed to apply each on in parallel.
if (toInstall.length) {
self._applyInstalls(toInstall, callback);
return;
}
callback();
});
return;
}
// If there were no uninstalls, only apply installs
if (toInstall.length) {
this._applyInstalls(toInstall, callback);
return;
}
this._log.info("There were no changes to be applied, returning");
callback();
return;
},
/**
* Apply a set of installs to the local registry. Fetch each app's manifest
* in parallel (don't retry on failure) and insert into registry.
*/
_applyInstalls: function _applyInstalls(toInstall, callback) {
let done = 0;
let total = toInstall.length;
this._log.info("Applying " + total + " installs to registry");
/**
* The purpose of _checkIfDone is to invoke the callback after all the
* installs have been applied. They all fire in parallel, and each will
* check-in when it is done.
*/
let self = this;
function _checkIfDone() {
done += 1;
self._log.debug(done + "/" + total + " apps processed");
if (done == total) {
callback();
}
}
function _makeManifestCallback(appObj) {
return function(err, manifest) {
if (err) {
self._log.warn("Could not fetch manifest for " + appObj.name);
_checkIfDone();
return;
}
appObj.value.manifest = manifest;
DOMApplicationRegistry.updateApps([appObj], _checkIfDone);
}
}
/**
* Now we get a manifest for each record, and add it to the local registry
* when we receive it. If a manifest GET times out, we will not add
* the app to the registry but count as "success" anyway. The app will
* be added on the next GET poll, hopefully the manifest will be
* available then.
*/
for each (let app in toInstall) {
let url = app.value.manifestURL;
if (url[0] == "/") {
url = app.value.origin + app.value.manifestURL;
}
this._getManifest(url, _makeManifestCallback(app));
}
},
/**
* Fetch a manifest from given URL. No retries are made on failure. We'll
* timeout after 20 seconds.
*/
_getManifest: function _getManifest(url, callback) {
let req = new RESTRequest(url);
req.timeout = 20;
let self = this;
req.get(function(error) {
if (error) {
callback(error, null);
return;
}
if (!req.response.success) {
callback(new Error("Non-200 while fetching manifest"), null);
return;
}
let err = null;
let manifest = null;
try {
manifest = JSON.parse(req.response.body);
if (!manifest.name) {
self._log.warn(
"_getManifest got invalid manifest: " + req.response.body
);
err = new Error("Invalid manifest fetched");
}
} catch (e) {
self._log.warn(
"_getManifest got invalid JSON response: " + req.response.body
);
err = new Error("Invalid manifest fetched");
}
callback(err, manifest);
});
},
};
XPCOMUtils.defineLazyGetter(this, "AitcStorage", function() {
return new AitcStorageImpl();
});

23
services/aitc/services-aitc.js Normal file
View File

@ -0,0 +1,23 @@
pref("services.aitc.browserid.url", "https://browserid.org/sign_in");
pref("services.aitc.browserid.log.level", "Debug");
pref("services.aitc.client.log.level", "Debug");
pref("services.aitc.client.timeout", 120); // 120 seconds
pref("services.aitc.dashboard.url", "https://myapps.mozillalabs.com");
pref("services.aitc.main.idleTime", 120000); // 2 minutes
pref("services.aitc.manager.putFreq", 10000); // 10 seconds
pref("services.aitc.manager.getActiveFreq", 120000); // 2 minutes
pref("services.aitc.manager.getPassiveFreq", 7200000); // 2 hours
pref("services.aitc.manager.log.level", "Debug");
pref("services.aitc.marketplace.url", "https://marketplace.mozilla.org");
pref("services.aitc.service.log.level", "Debug");
// Temporary value. Change to the production server when we get the OK from server ops
pref("services.aitc.tokenServer.url", "https://stage-token.services.mozilla.com");
pref("services.aitc.storage.log.level", "Debug");

25
services/aitc/tests/Makefile.in Normal file
View File

@ -0,0 +1,25 @@
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this file,
# You can obtain one at http://mozilla.org/MPL/2.0/.
DEPTH = ../../..
topsrcdir = @top_srcdir@
srcdir = @srcdir@
VPATH = @srcdir@
relativesrcdir = services/aitc/tests
include $(DEPTH)/config/autoconf.mk
MODULE = test_services_aitc
XPCSHELL_TESTS = unit
include $(topsrcdir)/config/rules.mk
_browser_files = \
mochitest/head.js \
mochitest/browser_id_simple.js \
mochitest/file_browser_id_mock.html \
$(NULL)
libs:: $(_browser_files)
$(INSTALL) $(foreach f,$^,"$f") $(DEPTH)/_tests/testing/mochitest/browser/$(relativesrcdir)

44
services/aitc/tests/mochitest/browser_id_simple.js Normal file
View File

@ -0,0 +1,44 @@
/* Any copyright is dedicated to the Public Domain.
http://creativecommons.org/publicdomain/zero/1.0/ */
const AUD = "http://foo.net";
function test() {
waitForExplicitFinish();
setEndpoint("browser_id_mock");
// Get an assertion for default email.
BrowserID.getAssertion(gotDefaultAssertion, {audience: AUD});
}
function gotDefaultAssertion(err, ast) {
is(err, null, "gotDefaultAssertion failed with " + err);
is(ast, "default@example.org_assertion_" + AUD,
"gotDefaultAssertion returned wrong assertion");
// Get an assertion for a specific email.
BrowserID.getAssertion(gotSpecificAssertion, {
requiredEmail: "specific@example.org",
audience: AUD
});
}
function gotSpecificAssertion(err, ast) {
is(err, null, "gotSpecificAssertion failed with " + err);
is(ast, "specific@example.org_assertion_" + AUD,
"gotSpecificAssertion returned wrong assertion");
// Get an assertion using sameEmailAs for another domain.
BrowserID.getAssertion(gotSameEmailAssertion, {
sameEmailAs: "http://zombo.com",
audience: AUD
});
}
function gotSameEmailAssertion(err, ast) {
is(err, null, "gotSameEmailAssertion failed with " + err);
is(ast, "assertion_for_sameEmailAs",
"gotSameEmailAssertion returned wrong assertion");
finish();
}

52
services/aitc/tests/mochitest/file_browser_id_mock.html Normal file
View File

@ -0,0 +1,52 @@
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p>Mock BrowserID endpoint for a logged-in user</p>
</body>
<script>
/**
* Object containing valid email/key paris for this user. An assertion is simply
* the string "_assertion_$audience" appended to the email. The exception is
* when the email address is "sameEmailAs@example.org" the assertion will
* be "assertion_for_sameEmailAs".
*/
var _emails = {
"default@example.org": "default@example.org_key",
"specific@example.org": "specific@example.org_key",
"sameEmailAs@example.org": "sameEmailAs@example.org_key"
};
var _sameEmailAs = "sameEmailAs@example.org";
// Mock internal API
window.BrowserID = {};
window.BrowserID.User = {
getStoredEmailKeypairs: function() {
return _emails;
},
getAssertion: function(email, audience, success, error) {
if (email == _sameEmailAs) {
success("assertion_for_sameEmailAs");
return;
}
if (email in _emails) {
success(email + "_assertion_" + audience);
return;
}
error("invalid email specified");
}
};
window.BrowserID.Storage = {
site: {
get: function(domain, key) {
if (key == "email") {
return _sameEmailAs;
}
return "";
}
}
};
</script>
</html>

23
services/aitc/tests/mochitest/head.js Normal file
View File

@ -0,0 +1,23 @@
/* Any copyright is dedicated to the Public Domain.
http://creativecommons.org/publicdomain/zero/1.0/ */
let tmp = {};
Components.utils.import("resource://gre/modules/Services.jsm");
Components.utils.import("resource://services-aitc/browserid.js", tmp);
const BrowserID = tmp.BrowserID;
const testPath = "http://mochi.test:8888/browser/services/aitc/tests/";
function loadURL(aURL, aCB) {
gBrowser.selectedBrowser.addEventListener("load", function () {
gBrowser.selectedBrowser.removeEventListener("load", arguments.callee, true);
is(gBrowser.currentURI.spec, aURL, "loaded expected URL");
aCB();
}, true);
gBrowser.loadURI(aURL);
}
function setEndpoint(name) {
let fullPath = testPath + "file_" + name + ".html";
Services.prefs.setCharPref("services.aitc.browserid.url", fullPath);
}

13
services/aitc/tests/unit/test_load_modules.js Normal file
View File

@ -0,0 +1,13 @@
const modules = [
"client.js",
"browserid.js",
"main.js",
"manager.js",
"storage.js"
];
function run_test() {
for each (let m in modules) {
Cu.import("resource://services-aitc/" + m, {});
}
}

117
services/aitc/tests/unit/test_storage_queue.js Normal file
View File

@ -0,0 +1,117 @@
/* Any copyright is dedicated to the Public Domain.
http://creativecommons.org/publicdomain/zero/1.0/ */
Cu.import("resource://services-aitc/storage.js");
Cu.import("resource://services-common/async.js");
let queue = null;
function run_test() {
queue = new AitcQueue("test", run_next_test);
}
add_test(function test_queue_create() {
do_check_eq(queue._queue.length, 0);
do_check_eq(queue._writeLock, false);
run_next_test();
});
add_test(function test_queue_enqueue() {
// Add to queue.
let testObj = {foo: "bar"};
queue.enqueue(testObj, function(err, done) {
do_check_eq(err, null);
do_check_true(done);
// Check if peek value is correct.
do_check_eq(queue.peek(), testObj);
// Peek should be idempotent.
do_check_eq(queue.peek(), testObj);
run_next_test();
});
});
add_test(function test_queue_dequeue() {
// Remove an item and see if queue is empty.
queue.dequeue(function(err, done) {
do_check_eq(err, null);
do_check_true(done);
do_check_eq(queue.length, 0);
try {
queue.peek();
} catch (e) {
do_check_eq(e.toString(), "Error: Queue is empty");
run_next_test();
}
});
});
add_test(function test_queue_multiaddremove() {
// Queues should handle objects, strings and numbers.
let items = [{test:"object"}, "teststring", 42];
// Two random numbers: how many items to queue and how many to remove.
let num = Math.floor(Math.random() * 100 + 1);
let rem = Math.floor(Math.random() * num + 1);
// First insert all the items we will remove later.
for (let i = 0; i < rem; i++) {
let ins = items[Math.round(Math.random() * 2)];
let cb = Async.makeSpinningCallback();
queue.enqueue(ins, cb);
do_check_true(cb.wait());
}
do_check_eq(queue.length, rem);
// Now insert the items we won't remove.
let check = [];
for (let i = 0; i < (num - rem); i++) {
check.push(items[Math.round(Math.random() * 2)]);
let cb = Async.makeSpinningCallback();
queue.enqueue(check[check.length - 1], cb);
do_check_true(cb.wait());
}
do_check_eq(queue.length, num);
// Now dequeue rem items.
for (let i = 0; i < rem; i++) {
let cb = Async.makeSpinningCallback();
queue.dequeue(cb);
do_check_true(cb.wait());
}
do_check_eq(queue.length, num - rem);
// Check that the items left are the right ones.
do_check_eq(JSON.stringify(queue._queue), JSON.stringify(check));
// Another instance of the same queue should return correct data.
let queue2 = new AitcQueue("test", function(done) {
do_check_true(done);
do_check_eq(queue2.length, queue.length);
do_check_eq(JSON.stringify(queue._queue), JSON.stringify(queue2._queue));
run_next_test();
});
});
/* TODO Bug 760905 - Temporarily disabled for orange.
add_test(function test_queue_writelock() {
// Queue should not enqueue or dequeue if lock is enabled.
queue._writeLock = true;
let len = queue.length;
queue.enqueue("writeLock test", function(err, done) {
do_check_eq(err.toString(), "Error: _putFile already in progress");
do_check_eq(queue.length, len);
queue.dequeue(function(err, done) {
do_check_eq(err.toString(), "Error: _putFile already in progress");
do_check_eq(queue.length, len);
run_next_test();
});
});
});
*/

124
services/aitc/tests/unit/test_storage_registry.js Normal file
View File

@ -0,0 +1,124 @@
/* Any copyright is dedicated to the Public Domain.
http://creativecommons.org/publicdomain/zero/1.0/ */
Cu.import("resource://gre/modules/Webapps.jsm");
Cu.import("resource://services-aitc/storage.js");
const START_PORT = 8080;
const SERVER = "http://localhost";
let fakeApp1 = {
origin: SERVER + ":" + START_PORT,
receipts: [],
manifestURL: "/manifest.webapp",
installOrigin: "http://localhost",
installedAt: Date.now(),
modifiedAt: Date.now()
};
// Valid manifest for app1
let manifest1 = {
name: "Appasaurus",
description: "Best fake app ever",
launch_path: "/",
fullscreen: true,
required_features: ["webgl"]
};
let fakeApp2 = {
origin: SERVER + ":" + (START_PORT + 1),
receipts: ["fake.jwt.token"],
manifestURL: "/manifest.webapp",
installOrigin: "http://localhost",
installedAt: Date.now(),
modifiedAt: Date.now()
};
// Invalid manifest for app2
let manifest2_bad = {
not: "a manifest",
fullscreen: true
};
// Valid manifest for app2
let manifest2_good = {
name: "Supercalifragilisticexpialidocious",
description: "Did we blow your mind yet?",
launch_path: "/"
};
let fakeApp3 = {
origin: SERVER + ":" + (START_PORT + 3), // 8082 is for the good manifest2
receipts: [],
manifestURL: "/manifest.webapp",
installOrigin: "http://localhost",
installedAt: Date.now(),
modifiedAt: Date.now()
};
let manifest3 = {
name: "Taumatawhakatangihangakoauauotamateapokaiwhenuakitanatahu",
description: "Find your way around this beautiful hill",
launch_path: "/"
};
function create_servers() {
// Setup servers to server manifests at each port
let manifests = [manifest1, manifest2_bad, manifest2_good, manifest3];
for (let i = 0; i < manifests.length; i++) {
let response = JSON.stringify(manifests[i]);
httpd_setup({"/manifest.webapp": function(req, res) {
res.setStatusLine(req.httpVersion, 200, "OK");
res.setHeader("Content-Type", "application/x-web-app-manifest+json");
res.bodyOutputStream.write(response, response.length);
}}, START_PORT + i);
}
}
function run_test() {
create_servers();
run_next_test();
}
add_test(function test_storage_install() {
let apps = [fakeApp1, fakeApp2];
AitcStorage.processApps(apps, function() {
// Verify that app1 got added to registry
let id = DOMApplicationRegistry._appId(fakeApp1.origin);
do_check_eq(DOMApplicationRegistry.itemExists(id), true);
// app2 should be missing because of bad manifest
do_check_eq(DOMApplicationRegistry._appId(fakeApp2.origin), null);
// Now associate fakeApp2 with a good manifest and process again
fakeApp2.origin = SERVER + ":8082";
AitcStorage.processApps([fakeApp1, fakeApp2], function() {
// Both apps must be installed
let id1 = DOMApplicationRegistry._appId(fakeApp1.origin);
let id2 = DOMApplicationRegistry._appId(fakeApp2.origin);
do_check_eq(DOMApplicationRegistry.itemExists(id1), true);
do_check_eq(DOMApplicationRegistry.itemExists(id2), true);
run_next_test();
});
});
});
add_test(function test_storage_uninstall() {
// Set app1 as deleted.
fakeApp1.deleted = true;
AitcStorage.processApps([fakeApp2], function() {
// It should be missing.
do_check_eq(DOMApplicationRegistry._appId(fakeApp1.origin), null);
run_next_test();
});
});
add_test(function test_storage_uninstall_empty() {
// Now remove app2 by virtue of it missing in the remote list.
AitcStorage.processApps([fakeApp3], function() {
let id3 = DOMApplicationRegistry._appId(fakeApp3.origin);
do_check_eq(DOMApplicationRegistry.itemExists(id3), true);
do_check_eq(DOMApplicationRegistry._appId(fakeApp2.origin), null);
run_next_test();
});
});

7
services/aitc/tests/unit/xpcshell.ini Normal file
View File

@ -0,0 +1,7 @@
[DEFAULT]
head = ../../../common/tests/unit/head_global.js ../../../common/tests/unit/head_helpers.js
tail =
[test_load_modules.js]
[test_storage_queue.js]
[test_storage_registry.js]

26
services/common/Makefile.in
View File

@ -31,4 +31,30 @@ libs::
TEST_DIRS += tests
# TODO enable once build infra supports testing modules.
#TESTING_JS_MODULES := aitcserver.js storageserver.js
#TESTING_JS_MODULE_DIR := services-common
# What follows is a helper to launch a standalone storage server instance.
# Most of the code lives in a Python script in the tests directory. If we
# ever consolidate our Python code, and/or have a supplemental driver for the
# build system, this can go away.
storage_server_hostname := localhost
storage_server_port := 8080
head_path = $(srcdir)/tests/unit
storage-server:
$(PYTHON) $(srcdir)/tests/run_server.py $(topsrcdir) \
$(MOZ_BUILD_ROOT) run_storage_server.js --port $(storage_server_port)
# And the same thing for an AITC server.
aitc_server_hostname := localhost
aitc_server_port := 8080
aitc-server:
$(PYTHON) $(srcdir)/tests/run_server.py $(topsrcdir) \
$(MOZ_BUILD_ROOT) run_aitc_server.js --port $(aitc_server_port)
include $(topsrcdir)/config/rules.mk

26
services/common/tests/run_aitc_server.js Normal file
View File

@ -0,0 +1,26 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this file,
* You can obtain one at http://mozilla.org/MPL/2.0/. */
/**
* This file runs a standalone AITC server.
*
* It is meant to be executed with an xpcshell.
*
* The Makefile in this directory contains a target to run it:
*
* $ make aitc-server
*/
Cu.import("resource://testing-common/services-common/aitcserver.js");
initTestLogging();
let server = new AITCServer10Server();
server.autoCreateUsers = true;
server.start(SERVER_PORT);
_("AITC server started on port " + SERVER_PORT);
// Launch the thread manager.
_do_main();

79
services/common/tests/run_server.py Executable file
View File

@ -0,0 +1,79 @@
#!/usr/bin/python
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this file,
# You can obtain one at http://mozilla.org/MPL/2.0/.
from argparse import ArgumentParser
from shutil import rmtree
from subprocess import Popen
from sys import argv
from sys import exit
from tempfile import mkdtemp
DEFAULT_PORT = 8080
DEFAULT_HOSTNAME = 'localhost'
def run_server(srcdir, objdir, js_file, hostname=DEFAULT_HOSTNAME,
port=DEFAULT_PORT):
dist_dir = '%s/dist' % objdir
head_dir = '%s/services/common/tests/unit' % srcdir
head_paths = [
'head_global.js',
'head_helpers.js',
'head_http.js',
]
head_paths = ['"%s/%s"' % (head_dir, path) for path in head_paths]
args = [
'%s/bin/xpcshell' % dist_dir,
'-g', '%s/bin' % dist_dir,
'-a', '%s/bin' % dist_dir,
'-r', '%s/bin/components/httpd.manifest' % dist_dir,
'-m',
'-n',
'-s',
'-f', '%s/testing/xpcshell/head.js' % srcdir,
'-e', 'const _SERVER_ADDR = "%s";' % hostname,
'-e', 'const _TESTING_MODULES_DIR = "%s/_tests/modules";' % objdir,
'-e', 'const SERVER_PORT = "%s";' % port,
'-e', 'const INCLUDE_FILES = [%s];' % ', '.join(head_paths),
'-e', '_register_protocol_handlers();',
'-e', 'for each (let name in INCLUDE_FILES) load(name);',
'-e', '_fakeIdleService.activate();',
'-f', '%s/services/common/tests/%s' % (srcdir, js_file)
]
profile_dir = mkdtemp()
print 'Created profile directory: %s' % profile_dir
try:
env = {'XPCSHELL_TEST_PROFILE_DIR': profile_dir}
proc = Popen(args, env=env)
return proc.wait()
finally:
print 'Removing profile directory %s' % profile_dir
rmtree(profile_dir)
if __name__ == '__main__':
parser = ArgumentParser(description="Run a standalone JS server.")
parser.add_argument('srcdir',
help="Root directory of Firefox source code.")
parser.add_argument('objdir',
help="Root directory object directory created during build.")
parser.add_argument('js_file',
help="JS file (in this directory) to execute.")
parser.add_argument('--port', default=DEFAULT_PORT, type=int,
help="Port to run server on.")
parser.add_argument('--address', default=DEFAULT_HOSTNAME,
help="Hostname to bind server to.")
args = parser.parse_args()
exit(run_server(args.srcdir, args.objdir, args.js_file, args.address,
args.port))

25
services/common/tests/run_storage_server.js Normal file
View File

@ -0,0 +1,25 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this file,
* You can obtain one at http://mozilla.org/MPL/2.0/. */
/**
* This file runs a Storage Service server.
*
* It is meant to be executed with an xpcshell.
*
* The Makefile in this directory contains a target to run it:
*
* $ make storage-server
*/
Cu.import("resource://testing-common/services-common/storageserver.js");
initTestLogging();
let server = new StorageServer();
server.allowAllUsers = true;
server.startSynchronous(SERVER_PORT);
_("Storage server started on port " + SERVER_PORT);
// Launch the thread manager.
_do_main();

528
services/common/tests/unit/aitcserver.js Normal file
View File

@ -0,0 +1,528 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this file,
* You can obtain one at http://mozilla.org/MPL/2.0/. */
"use strict";
// TODO enable once build infra supports test modules.
/*
const {classes: Cc, interfaces: Ci, utils: Cu} = Components;
const EXPORTED_SYMBOLS = [
"AITCServer10User",
"AITCServer10Server",
];
Cu.import("resource://testing-common/httpd.js");
*/
Cu.import("resource://services-crypto/utils.js");
Cu.import("resource://services-common/log4moz.js");
Cu.import("resource://services-common/utils.js");
/**
* Represents an individual user on an AITC 1.0 server.
*
* This type provides convenience APIs for interacting with an individual
* user's data.
*/
function AITCServer10User() {
this._log = Log4Moz.repository.getLogger("Services.Common.AITCServer");
this.apps = {};
}
AITCServer10User.prototype = {
appRecordProperties: {
origin: true,
manifestPath: true,
installOrigin: true,
installedAt: true,
modifiedAt: true,
receipts: true,
name: true,
deleted: true,
},
requiredAppProperties: [
"origin",
"manifestPath",
"installOrigin",
"installedAt",
"modifiedAt",
"name",
"receipts",
],
/**
* Obtain the apps for this user.
*
* This is a generator of objects representing the apps. Returns the original
* apps object normally or an abbreviated version if `minimal` is truthy.
*/
getApps: function getApps(minimal) {
let result;
for (let id in this.apps) {
let app = this.apps[id];
if (!minimal) {
yield app;
continue;
}
yield {origin: app.origin, modifiedAt: app.modifiedAt};
}
},
getAppByID: function getAppByID(id) {
return this.apps[id];
},
/**
* Adds an app to this user.
*
* The app record should be an object (likely from decoded JSON).
*/
addApp: function addApp(app) {
for (let k in app) {
if (!(k in this.appRecordProperties)) {
throw new Error("Unexpected property in app record: " + k);
}
}
for each (let k in this.requiredAppProperties) {
if (!(k in app)) {
throw new Error("Required property not in app record: " + k);
}
}
this.apps[this.originToID(app.origin)] = app;
},
/**
* Returns whether a user has an app with the specified ID.
*/
hasAppID: function hasAppID(id) {
return id in this.apps;
},
/**
* Delete an app having the specified ID.
*/
deleteAppWithID: function deleteAppWithID(id) {
delete this.apps[id];
},
/**
* Convert an origin string to an ID.
*/
originToID: function originToID(origin) {
let hash = CryptoUtils.UTF8AndSHA1(origin);
return CommonUtils.encodeBase64URL(hash, false);
},
};
/**
* A fully-functional AITC 1.0 server implementation.
*
* Each server instance is capable of serving requests for multiple users.
* By default, users do not exist and requests to URIs for a specific user
* will result in 404s. To register a new user with an empty account, call
* createUser(). If you wish for HTTP requests for non-existing users to
* work, set autoCreateUsers to true and am empty user will be
* provisioned at request time.
*/
function AITCServer10Server() {
this._log = Log4Moz.repository.getLogger("Services.Common.AITCServer");
this.server = new nsHttpServer();
this.port = null;
this.users = {};
this.autoCreateUsers = false;
this._appsAppHandlers = {
GET: this._appsAppGetHandler,
PUT: this._appsAppPutHandler,
DELETE: this._appsAppDeleteHandler,
};
}
AITCServer10Server.prototype = {
ID_REGEX: /^[a-zA-Z0-9_-]{27}$/,
VERSION_PATH: "/1.0/",
/**
* Obtain the base URL the server can be accessed at as a string.
*/
get url() {
// Is this available on the nsHttpServer instance?
return "http://localhost:" + this.port + this.VERSION_PATH;
},
/**
* Start the server on a specified port.
*/
start: function start(port) {
if (!port) {
throw new Error("port argument must be specified.");
}
this.port = port;
this.server.registerPrefixHandler(this.VERSION_PATH,
this._generalHandler.bind(this));
this.server.start(port);
},
/**
* Stop the server.
*
* Calls the specified callback when the server is stopped.
*/
stop: function stop(cb) {
let handler = {onStopped: cb};
this.server.stop(handler);
},
createUser: function createUser(username) {
if (username in this.users) {
throw new Error("User already exists: " + username);
}
this._log.info("Registering user: " + username);
this.users[username] = new AITCServer10User();
this.server.registerPrefixHandler(this.VERSION_PATH + username + "/",
this._userHandler.bind(this, username));
return this.users[username];
},
/**
* Returns information for an individual user.
*
* The returned object contains functions to access and manipulate an
* individual user.
*/
getUser: function getUser(username) {
if (!(username in this.users)) {
throw new Error("user is not present in server: " + username);
}
return this.users[username];
},
/**
* HTTP handler for requests to /1.0/ which don't have a specific user
* registered.
*/
_generalHandler: function _generalHandler(request, response) {
let path = request.path;
this._log.info("Request: " + request.method + " " + path);
if (path.indexOf(this.VERSION_PATH) != 0) {
throw new Error("generalHandler invoked improperly.");
}
let rest = request.path.substr(this.VERSION_PATH.length);
if (!rest.length) {
throw HTTP_404;
}
if (!this.autoCreateUsers) {
throw HTTP_404;
}
let username;
let index = rest.indexOf("/");
if (index == -1) {
username = rest;
} else {
username = rest.substr(0, index);
}
this.createUser(username);
this._userHandler(username, request, response);
},
/**
* HTTP handler for requests for a specific user.
*
* This handles request routing to the appropriate handler.
*/
_userHandler: function _userHandler(username, request, response) {
this._log.info("Request: " + request.method + " " + request.path);
let path = request.path;
let prefix = this.VERSION_PATH + username + "/";
if (path.indexOf(prefix) != 0) {
throw new Error("userHandler invoked improperly.");
}
let user = this.users[username];
if (!user) {
throw new Error("User handler should not have been invoked for an " +
"unknown user!");
}
let requestTime = Date.now();
response.dispatchTime = requestTime;
response.setHeader("X-Timestamp", "" + requestTime);
let handler;
let remaining = path.substr(prefix.length);
if (remaining == "apps" || remaining == "apps/") {
this._log.info("Dispatching to apps index handler.");
handler = this._appsIndexHandler.bind(this, user, request, response);
} else if (!remaining.indexOf("apps/")) {
let id = remaining.substr("apps/".length);
this._log.info("Dispatching to app handler.");
handler = this._appsAppHandler.bind(this, user, id, request, response);
} else if (remaining == "devices" || !remaining.indexOf("devices/")) {
this._log.info("Dispatching to devices handler.");
handler = this._devicesHandler.bind(this, user,
remaining.substr("devices".length),
request, response);
} else {
throw HTTP_404;
}
try {
handler();
} catch (ex) {
if (ex instanceof HttpError) {
response.setStatusLine(request.httpVersion, ex.code, ex.description);
return;
}
this._log.warn("Exception when processing request: " +
CommonUtils.exceptionStr(ex));
throw ex;
}
},
_appsIndexHandler: function _appsIndexHandler(user, request, response) {
if (request.method != "GET") {
response.setStatusLine(request.httpVersion, 405, "Method Not Allowed");
response.setHeader("Accept", "GET");
return;
}
let options = this._getQueryStringParams(request);
for (let key in options) {
let value = options[key];
switch (key) {
case "after":
let time = parseInt(value, 10);
if (isNaN(time)) {
throw HTTP_400;
}
options.after = time;
break;
case "full":
// Value is irrelevant.
break;
default:
this._log.info("Unknown query string parameter: " + key);
throw HTTP_400;
}
}
let apps = [];
let newest = 0;
for each (let app in user.getApps(!("full" in options))) {
if (app.modifiedAt > newest) {
newest = app.modifiedAt;
}
if ("after" in options && app.modifiedAt <= options.after) {
continue;
}
apps.push(app);
}
if (request.hasHeader("X-If-Modified-Since")) {
let modified = parseInt(request.getHeader("X-If-Modified-Since"), 10);
if (modified >= newest) {
response.setStatusLine(request.httpVersion, 304, "Not Modified");
return;
}
}
let body = JSON.stringify({apps: apps});
response.setStatusLine(request.httpVersion, 200, "OK");
response.setHeader("X-Last-Modified", "" + newest);
response.setHeader("Content-Type", "application/json");
response.bodyOutputStream.write(body, body.length);
},
_appsAppHandler: function _appAppHandler(user, id, request, response) {
if (!(request.method in this._appsAppHandlers)) {
response.setStatusLine(request.httpVersion, 405, "Method Not Allowed");
response.setHeader("Accept", Object.keys(this._appsAppHandlers).join(","));
return;
}
let handler = this._appsAppHandlers[request.method];
return handler.call(this, user, id, request, response);
},
_appsAppGetHandler: function _appsAppGetHandler(user, id, request, response) {
if (!user.hasAppID(id)) {
throw HTTP_404;
}
let app = user.getAppByID(id);
if (request.hasHeader("X-If-Modified-Since")) {
let modified = parseInt(request.getHeader("X-If-Modified-Since"), 10);
this._log.debug("Client time: " + modified + "; Server time: " +
app.modifiedAt);
if (modified >= app.modifiedAt) {
response.setStatusLine(request.httpVersion, 304, "Not Modified");
return;
}
}
let body = JSON.stringify(app);
response.setStatusLine(request.httpVersion, 200, "OK");
response.setHeader("X-Last-Modified", "" + response.dispatchTime);
response.setHeader("Content-Type", "application/json");
response.bodyOutputStream.write(body, body.length);
},
_appsAppPutHandler: function _appsAppPutHandler(user, id, request, response) {
if (!request.hasHeader("Content-Type")) {
this._log.info("Request does not have Content-Type header.");
throw HTTP_400;
}
let ct = request.getHeader("Content-Type");
if (ct != "application/json" && ct.indexOf("application/json;") !== 0) {
this._log.info("Unknown media type: " + ct);
// TODO proper response headers.
throw HTTP_415;
}
let requestBody = CommonUtils.readBytesFromInputStream(request.bodyInputStream);
this._log.debug("Request body: " + requestBody);
if (requestBody.length > 8192) {
this._log.info("Request body too long: " + requestBody.length);
throw HTTP_413;
}
let hadApp = user.hasAppID(id);
let app;
try {
app = JSON.parse(requestBody);
} catch (e) {
this._log.info("JSON parse error.");
throw HTTP_400;
}
// URL and record mismatch.
if (user.originToID(app.origin) != id) {
this._log.warn("URL ID and origin mismatch. URL: " + id + "; Record: " +
user.originToID(app.origin));
throw HTTP_403;
}
if (request.hasHeader("X-If-Unmodified-Since") && hadApp) {
let modified = parseInt(request.getHeader("X-If-Unmodified-Since"), 10);
let existing = user.getAppByID(id);
if (existing.modifiedAt > modified) {
this._log.info("Server modified after client.");
throw HTTP_412;
}
}
try {
app.modifiedAt = response.dispatchTime;
if (hadApp) {
app.installedAt = user.getAppByID(id).installedAt;
} else {
app.installedAt = response.dispatchTime;
}
user.addApp(app);
} catch (e) {
this._log.info("Error adding app: " + CommonUtils.exceptionStr(e));
throw HTTP_400;
}
let code = 201;
let status = "Created";
if (hadApp) {
code = 204;
status = "No Content";
}
response.setHeader("X-Last-Modified", "" + response.dispatchTime);
response.setStatusLine(request.httpVersion, code, status);
},
_appsAppDeleteHandler: function _appsAppDeleteHandler(user, id, request,
response) {
if (!user.hasAppID(id)) {
throw HTTP_404;
}
let existing = user.getAppByID(id);
if (request.hasHeader("X-If-Unmodified-Since")) {
let modified = parseInt(request.getHeader("X-If-Unmodified-Since"), 10);
if (existing.modifiedAt > modified) {
throw HTTP_412;
}
}
user.deleteAppWithID(id);
response.setHeader("X-Last-Modified", "" + response.dispatchTime);
response.setStatusLine(request.httpVersion, 204, "No Content");
},
_devicesHandler: function _devicesHandler(user, path, request, response) {
// TODO need to support full API.
// For now, we just assume it is a request for /.
response.setHeader("Content-Type", "application/json");
let body = JSON.stringify({devices: []});
response.setStatusLine(request.httpVersion, 200, "OK");
response.bodyOutputStream.write(body, body.length);
},
// Surely this exists elsewhere in the Mozilla source tree...
_getQueryStringParams: function _getQueryStringParams(request) {
let params = {};
for each (let chunk in request.queryString.split("&")) {
if (!chunk) {
continue;
}
let parts = chunk.split("=");
// TODO URL decode key and value.
if (parts.length == 1) {
params[parts[0]] = "";
} else {
params[parts[0]] = parts[1];
}
}
return params;
},
};

2
services/common/tests/unit/head_global.js
View File

@ -47,7 +47,7 @@ function addResourceAlias() {
const handler = Services.io.getProtocolHandler("resource")
.QueryInterface(Ci.nsIResProtocolHandler);
let modules = ["common", "crypto"];
let modules = ["aitc", "common", "crypto"];
for each (let module in modules) {
let uri = Services.io.newURI("resource:///modules/services-" + module + "/",
null, null);

32
services/common/tests/unit/head_helpers.js
View File

@ -3,6 +3,7 @@
* You can obtain one at http://mozilla.org/MPL/2.0/. */
Cu.import("resource://services-common/log4moz.js");
Cu.import("resource://services-common/utils.js");
let btoa = Cu.import("resource://services-common/log4moz.js").btoa;
let atob = Cu.import("resource://services-common/log4moz.js").atob;
@ -57,13 +58,15 @@ function initTestLogging(level) {
};
LogStats.prototype.__proto__ = new Log4Moz.Formatter();
var log = Log4Moz.repository.rootLogger;
var logStats = new LogStats();
var appender = new Log4Moz.DumpAppender(logStats);
let log = Log4Moz.repository.rootLogger;
let logStats = new LogStats();
let appender = new Log4Moz.DumpAppender(logStats);
if (typeof(level) == "undefined")
if (typeof(level) == "undefined") {
level = "Debug";
}
getTestLogger().level = Log4Moz.Level[level];
Log4Moz.repository.getLogger("Services").level = Log4Moz.Level[level];
log.level = Log4Moz.Level.Trace;
appender.level = Log4Moz.Level.Trace;
@ -78,6 +81,16 @@ function getTestLogger(component) {
return Log4Moz.repository.getLogger("Testing");
}
/**
* Obtain a port number to run a server on.
*
* In the ideal world, this would be dynamic so multiple servers could be run
* in parallel.
*/
function get_server_port() {
return 8080;
}
function httpd_setup (handlers, port) {
let port = port || 8080;
let server = new nsHttpServer();
@ -89,7 +102,7 @@ function httpd_setup (handlers, port) {
} catch (ex) {
_("==========================================");
_("Got exception starting HTTP server on port " + port);
_("Error: " + Utils.exceptionStr(ex));
_("Error: " + CommonUtils.exceptionStr(ex));
_("Is there a process already listening on port " + port + "?");
_("==========================================");
do_throw(ex);
@ -117,14 +130,7 @@ function httpd_handler(statusCode, status, body) {
* all available input is read.
*/
function readBytesFromInputStream(inputStream, count) {
var BinaryInputStream = Components.Constructor(
"@mozilla.org/binaryinputstream;1",
"nsIBinaryInputStream",
"setInputStream");
if (!count) {
count = inputStream.available();
}
return new BinaryInputStream(inputStream).readBytes(count);
return CommonUtils.readBytesFromInputStream(inputStream, count);
}
/*

29
services/common/tests/unit/head_http.js Normal file
View File

@ -0,0 +1,29 @@
/* Any copyright is dedicated to the Public Domain.
http://creativecommons.org/publicdomain/zero/1.0/ */
Cu.import("resource://services-common/utils.js");
function basic_auth_header(user, password) {
return "Basic " + btoa(user + ":" + CommonUtils.encodeUTF8(password));
}
function basic_auth_matches(req, user, password) {
if (!req.hasHeader("Authorization")) {
return false;
}
let expected = basic_auth_header(user, CommonUtils.encodeUTF8(password));
return req.getHeader("Authorization") == expected;
}
function httpd_basic_auth_handler(body, metadata, response) {
if (basic_auth_matches(metadata, "guest", "guest")) {
response.setStatusLine(metadata.httpVersion, 200, "OK, authorized");
response.setHeader("WWW-Authenticate", 'Basic realm="secret"', false);
} else {
body = "This path exists and is protected - failed";
response.setStatusLine(metadata.httpVersion, 401, "Unauthorized");
response.setHeader("WWW-Authenticate", 'Basic realm="secret"', false);
}
response.bodyOutputStream.write(body, body.length);
}

1638
services/common/tests/unit/storageserver.js Normal file
View File

File diff suppressed because it is too large Load Diff

169
services/common/tests/unit/test_aitc_server.js Normal file
View File

@ -0,0 +1,169 @@
/* Any copyright is dedicated to the Public Domain.
http://creativecommons.org/publicdomain/zero/1.0/ */
"use strict";
Cu.import("resource://services-common/rest.js");
Cu.import("resource://services-common/utils.js");
// TODO enable once build infra supports testing modules.
//Cu.import("resource://testing-common/services-common/aitcserver.js");
function run_test() {
initTestLogging("Trace");
run_next_test();
}
function get_aitc_server() {
let server = new AITCServer10Server();
server.start(get_server_port());
return server;
}
function get_server_with_user(username) {
let server = get_aitc_server();
server.createUser(username);
return server;
}
add_test(function test_origin_conversion() {
let mapping = {
"www.mozilla.org": "xSMmiFEpg4b4TRtzJZd6Mvy4hGc",
"foo": "C-7Hteo_D9vJXQ3UfzxbwnXaijM",
};
for (let k in mapping) {
do_check_eq(AITCServer10User.prototype.originToID(k), mapping[k]);
}
run_next_test();
});
add_test(function test_empty_user() {
_("Ensure user instances can be created.");
let user = new AITCServer10User();
let apps = user.getApps();
do_check_eq([app for (app in apps)].length, 0);
do_check_false(user.hasAppID("foobar"));
run_next_test();
});
add_test(function test_user_add_app() {
_("Ensure apps can be added to users.");
let user = new AITCServer10User();
let threw = false;
try {
user.addApp({});
} catch (ex) {
threw = true;
} finally {
do_check_true(threw);
threw = false;
}
run_next_test();
});
add_test(function test_server_run() {
_("Ensure server can be started properly.");
let server = new AITCServer10Server();
server.start(get_server_port());
server.stop(run_next_test);
});
add_test(function test_create_user() {
_("Ensure users can be created properly.");
let server = get_aitc_server();
let u1 = server.createUser("123");
do_check_true(u1 instanceof AITCServer10User);
let u2 = server.getUser("123");
do_check_eq(u1, u2);
server.stop(run_next_test);
});
add_test(function test_empty_server_404() {
_("Ensure empty server returns 404.");
let server = get_aitc_server();
let request = new RESTRequest(server.url + "123/");
request.get(function onComplete(error) {
do_check_eq(this.response.status, 404);
let request = new RESTRequest(server.url + "123/apps/");
request.get(function onComplete(error) {
do_check_eq(this.response.status, 404);
server.stop(run_next_test);
});
});
});
add_test(function test_empty_user_apps() {
_("Ensure apps request for empty user has appropriate content.");
const username = "123";
let server = get_server_with_user(username);
let request = new RESTRequest(server.url + username + "/apps/");
_("Performing request...");
request.get(function onComplete(error) {
_("Got response");
do_check_eq(error, null);
do_check_eq(200, this.response.status);
let headers = this.response.headers;
do_check_true("content-type" in headers);
do_check_eq(headers["content-type"], "application/json");
do_check_true("x-timestamp" in headers);
let body = this.response.body;
let parsed = JSON.parse(body);
do_check_attribute_count(parsed, 1);
do_check_true("apps" in parsed);
do_check_true(Array.isArray(parsed.apps));
do_check_eq(parsed.apps.length, 0);
server.stop(run_next_test);
});
});
add_test(function test_invalid_request_method() {
_("Ensure HTTP 405 works as expected.");
const username = "12345";
let server = get_server_with_user(username);
let request = new RESTRequest(server.url + username + "/apps/foobar");
request.dispatch("SILLY", null, function onComplete(error) {
do_check_eq(error, null);
do_check_eq(this.response.status, 405);
let headers = this.response.headers;
do_check_true("accept" in headers);
let allowed = new Set();
for (let method of headers["accept"].split(",")) {
allowed.add(method);
}
do_check_eq(allowed.size(), 3);
for (let method of ["GET", "PUT", "DELETE"]) {
do_check_true(allowed.has(method));
}
run_next_test();
});
});

13
services/common/tests/unit/test_load_modules.js
View File

@ -11,9 +11,22 @@ const modules = [
"utils.js",
];
const test_modules = [
"aitcserver.js",
"storageserver.js",
];
function run_test() {
for each (let m in modules) {
let resource = "resource://services-common/" + m;
Components.utils.import(resource, {});
}
// TODO enable once build infra supports testing modules.
/*
for each (let m in test_modules) {
let resource = "resource://testing-common/services-common/" + m;
Components.utils.import(resource, {});
}
*/
}

517
services/common/tests/unit/test_storage_server.js Normal file
View File

@ -0,0 +1,517 @@
/* Any copyright is dedicated to the Public Domain.
http://creativecommons.org/publicdomain/zero/1.0/ */
Cu.import("resource://services-common/async.js");
Cu.import("resource://services-common/rest.js");
Cu.import("resource://services-common/utils.js");
// TODO enable once build infra supports testing modules.
//Cu.import("resource://testing-common/services-common/storageserver.js");
const PORT = 8080;
const DEFAULT_USER = "123";
const DEFAULT_PASSWORD = "password";
/**
* Helper function to prepare a RESTRequest against the server.
*/
function localRequest(path, user=DEFAULT_USER, password=DEFAULT_PASSWORD) {
_("localRequest: " + path);
let url = "http://127.0.0.1:" + PORT + path;
_("url: " + url);
let req = new RESTRequest(url);
let header = basic_auth_header(user, password);
req.setHeader("Authorization", header);
req.setHeader("Accept", "application/json");
return req;
}
/**
* Helper function to validate an HTTP response from the server.
*/
function validateResponse(response) {
do_check_true("x-timestamp" in response.headers);
if ("content-length" in response.headers) {
let cl = parseInt(response.headers["content-length"]);
if (cl != 0) {
do_check_true("content-type" in response.headers);
do_check_eq("application/json", response.headers["content-type"]);
}
}
if (response.status == 204 || response.status == 304) {
do_check_false("content-type" in response.headers);
if ("content-length" in response.headers) {
do_check_eq(response.headers["content-length"], "0");
}
}
if (response.status == 405) {
do_check_true("allow" in response.headers);
}
}
/**
* Helper function to synchronously wait for a response and validate it.
*/
function waitAndValidateResponse(cb, request) {
let error = cb.wait();
if (!error) {
validateResponse(request.response);
}
return error;
}
/**
* Helper function to synchronously perform a GET request.
*
* @return Error instance or null if no error.
*/
function doGetRequest(request) {
let cb = Async.makeSpinningCallback();
request.get(cb);
return waitAndValidateResponse(cb, request);
}
/**
* Helper function to synchronously perform a PUT request.
*
* @return Error instance or null if no error.
*/
function doPutRequest(request, data) {
let cb = Async.makeSpinningCallback();
request.put(data, cb);
return waitAndValidateResponse(cb, request);
}
/**
* Helper function to synchronously perform a DELETE request.
*
* @return Error or null if no error was encountered.
*/
function doDeleteRequest(request) {
let cb = Async.makeSpinningCallback();
request.delete(cb);
return waitAndValidateResponse(cb, request);
}
function run_test() {
Log4Moz.repository.getLogger("Services.Common.Test.StorageServer").level =
Log4Moz.Level.Trace;
initTestLogging();
run_next_test();
}
add_test(function test_creation() {
_("Ensure a simple server can be created.");
// Explicit callback for this one.
let server = new StorageServer({
__proto__: StorageServerCallback,
});
do_check_true(!!server);
server.start(PORT, function () {
_("Started on " + server.port);
do_check_eq(server.port, PORT);
server.stop(run_next_test);
});
});
add_test(function test_synchronous_start() {
_("Ensure starting using startSynchronous works.");
let server = new StorageServer();
server.startSynchronous(PORT);
server.stop(run_next_test);
});
add_test(function test_url_parsing() {
_("Ensure server parses URLs properly.");
let server = new StorageServer();
// Check that we can parse a BSO URI.
let parts = server.pathRE.exec("/2.0/12345/storage/crypto/keys");
let [all, version, user, first, rest] = parts;
do_check_eq(all, "/2.0/12345/storage/crypto/keys");
do_check_eq(version, "2.0");
do_check_eq(user, "12345");
do_check_eq(first, "storage");
do_check_eq(rest, "crypto/keys");
do_check_eq(null, server.pathRE.exec("/nothing/else"));
// Check that we can parse a collection URI.
parts = server.pathRE.exec("/2.0/123/storage/crypto");
let [all, version, user, first, rest] = parts;
do_check_eq(all, "/2.0/123/storage/crypto");
do_check_eq(version, "2.0");
do_check_eq(user, "123");
do_check_eq(first, "storage");
do_check_eq(rest, "crypto");
// We don't allow trailing slash on storage URI.
parts = server.pathRE.exec("/2.0/1234/storage/");
do_check_eq(parts, undefined);
// storage alone is a valid request.
parts = server.pathRE.exec("/2.0/123456/storage");
let [all, version, user, first, rest] = parts;
do_check_eq(all, "/2.0/123456/storage");
do_check_eq(version, "2.0");
do_check_eq(user, "123456");
do_check_eq(first, "storage");
do_check_eq(rest, undefined);
parts = server.storageRE.exec("storage");
let [all, storage, collection, id] = parts;
do_check_eq(all, "storage");
do_check_eq(collection, undefined);
run_next_test();
});
add_test(function test_basic_http() {
let server = new StorageServer();
server.registerUser("345", "password");
do_check_true(server.userExists("345"));
server.startSynchronous(PORT);
_("Started on " + server.port);
let req = localRequest("/2.0/storage/crypto/keys");
_("req is " + req);
req.get(function (err) {
do_check_eq(null, err);
server.stop(run_next_test);
});
});
add_test(function test_info_collections() {
let server = new StorageServer();
server.registerUser("123", "password");
server.startSynchronous(PORT);
let path = "/2.0/123/info/collections";
_("info/collections on empty server should be empty object.");
let request = localRequest(path, "123", "password");
let error = doGetRequest(request);
do_check_eq(error, null);
do_check_eq(request.response.status, 200);
do_check_eq(request.response.body, "{}");
_("Creating an empty collection should result in collection appearing.");
let coll = server.createCollection("123", "col1");
let request = localRequest(path, "123", "password");
let error = doGetRequest(request);
do_check_eq(error, null);
do_check_eq(request.response.status, 200);
let info = JSON.parse(request.response.body);
do_check_attribute_count(info, 1);
do_check_true("col1" in info);
do_check_eq(info.col1, coll.timestamp);
server.stop(run_next_test);
});
add_test(function test_bso_get_existing() {
_("Ensure that BSO retrieval works.");
let server = new StorageServer();
server.registerUser("123", "password");
server.createContents("123", {
test: {"bso": {"foo": "bar"}}
});
server.startSynchronous(PORT);
let coll = server.user("123").collection("test");
let request = localRequest("/2.0/123/storage/test/bso", "123", "password");
let error = doGetRequest(request);
do_check_eq(error, null);
do_check_eq(request.response.status, 200);
do_check_eq(request.response.headers["content-type"], "application/json");
let bso = JSON.parse(request.response.body);
do_check_attribute_count(bso, 3);
do_check_eq(bso.id, "bso");
do_check_eq(bso.modified, coll.bso("bso").modified);
let payload = JSON.parse(bso.payload);
do_check_attribute_count(payload, 1);
do_check_eq(payload.foo, "bar");
server.stop(run_next_test);
});
add_test(function test_bso_404() {
_("Ensure the server responds with a 404 if a BSO does not exist.");
let server = new StorageServer();
server.registerUser("123", "password");
server.createContents("123", {
test: {}
});
server.startSynchronous(PORT);
let request = localRequest("/2.0/123/storage/test/foo");
let error = doGetRequest(request);
do_check_eq(error, null);
do_check_eq(request.response.status, 404);
do_check_false("content-type" in request.response.headers);
server.stop(run_next_test);
});
add_test(function test_bso_if_modified_since_304() {
_("Ensure the server responds properly to X-If-Modified-Since for BSOs.");
let server = new StorageServer();
server.registerUser("123", "password");
server.createContents("123", {
test: {bso: {foo: "bar"}}
});
server.startSynchronous(PORT);
let coll = server.user("123").collection("test");
do_check_neq(coll, null);
// Rewind clock just in case.
coll.timestamp -= 10000;
coll.bso("bso").modified -= 10000;
let request = localRequest("/2.0/123/storage/test/bso", "123", "password");
request.setHeader("X-If-Modified-Since", "" + server.serverTime());
let error = doGetRequest(request);
do_check_eq(null, error);
do_check_eq(request.response.status, 304);
do_check_false("content-type" in request.response.headers);
let request = localRequest("/2.0/123/storage/test/bso", "123", "password");
request.setHeader("X-If-Modified-Since", "" + (server.serverTime() - 20000));
let error = doGetRequest(request);
do_check_eq(null, error);
do_check_eq(request.response.status, 200);
do_check_eq(request.response.headers["content-type"], "application/json");
server.stop(run_next_test);
});
add_test(function test_bso_if_unmodified_since() {
_("Ensure X-If-Unmodified-Since works properly on BSOs.");
let server = new StorageServer();
server.registerUser("123", "password");
server.createContents("123", {
test: {bso: {foo: "bar"}}
});
server.startSynchronous(PORT);
let coll = server.user("123").collection("test");
do_check_neq(coll, null);
let time = coll.timestamp;
_("Ensure we get a 412 for specified times older than server time.");
let request = localRequest("/2.0/123/storage/test/bso", "123", "password");
request.setHeader("X-If-Unmodified-Since", time - 5000);
request.setHeader("Content-Type", "application/json");
let payload = JSON.stringify({"payload": "foobar"});
let error = doPutRequest(request, payload);
do_check_eq(null, error);
do_check_eq(request.response.status, 412);
_("Ensure we get a 204 if update goes through.");
let request = localRequest("/2.0/123/storage/test/bso", "123", "password");
request.setHeader("Content-Type", "application/json");
request.setHeader("X-If-Unmodified-Since", time + 1);
let error = doPutRequest(request, payload);
do_check_eq(null, error);
do_check_eq(request.response.status, 204);
do_check_true(coll.timestamp > time);
// Not sure why a client would send X-If-Unmodified-Since if a BSO doesn't
// exist. But, why not test it?
_("Ensure we get a 201 if creation goes through.");
let request = localRequest("/2.0/123/storage/test/none", "123", "password");
request.setHeader("Content-Type", "application/json");
request.setHeader("X-If-Unmodified-Since", time);
let error = doPutRequest(request, payload);
do_check_eq(null, error);
do_check_eq(request.response.status, 201);
server.stop(run_next_test);
});
add_test(function test_bso_delete_not_exist() {
_("Ensure server behaves properly when deleting a BSO that does not exist.");
let server = new StorageServer();
server.registerUser("123", "password");
server.user("123").createCollection("empty");
server.startSynchronous(PORT);
server.callback.onItemDeleted = function onItemDeleted(username, collection,
id) {
do_throw("onItemDeleted should not have been called.");
};
let request = localRequest("/2.0/123/storage/empty/nada", "123", "password");
let error = doDeleteRequest(request);
do_check_eq(error, null);
do_check_eq(request.response.status, 404);
do_check_false("content-type" in request.response.headers);
server.stop(run_next_test);
});
add_test(function test_bso_delete_exists() {
_("Ensure proper semantics when deleting a BSO that exists.");
let server = new StorageServer();
server.registerUser("123", "password");
server.startSynchronous(PORT);
let coll = server.user("123").createCollection("test");
let bso = coll.insert("myid", {foo: "bar"});
let timestamp = coll.timestamp;
server.callback.onItemDeleted = function onDeleted(username, collection, id) {
delete server.callback.onItemDeleted;
do_check_eq(username, "123");
do_check_eq(collection, "test");
do_check_eq(id, "myid");
};
let request = localRequest("/2.0/123/storage/test/myid", "123", "password");
let error = doDeleteRequest(request);
do_check_eq(error, null);
do_check_eq(request.response.status, 204);
do_check_eq(coll.bsos().length, 0);
do_check_true(coll.timestamp > timestamp);
_("On next request the BSO should not exist.");
let request = localRequest("/2.0/123/storage/test/myid", "123", "password");
let error = doGetRequest(request);
do_check_eq(error, null);
do_check_eq(request.response.status, 404);
server.stop(run_next_test);
});
add_test(function test_bso_delete_unmodified() {
_("Ensure X-If-Unmodified-Since works when deleting BSOs.");
let server = new StorageServer();
server.startSynchronous(PORT);
server.registerUser("123", "password");
let coll = server.user("123").createCollection("test");
let bso = coll.insert("myid", {foo: "bar"});
let modified = bso.modified;
_("Issuing a DELETE with an older time should fail.");
let path = "/2.0/123/storage/test/myid";
let request = localRequest(path, "123", "password");
request.setHeader("X-If-Unmodified-Since", modified - 1000);
let error = doDeleteRequest(request);
do_check_eq(error, null);
do_check_eq(request.response.status, 412);
do_check_false("content-type" in request.response.headers);
do_check_neq(coll.bso("myid"), null);
_("Issuing a DELETE with a newer time should work.");
let request = localRequest(path, "123", "password");
request.setHeader("X-If-Unmodified-Since", modified + 1000);
let error = doDeleteRequest(request);
do_check_eq(error, null);
do_check_eq(request.response.status, 204);
do_check_true(coll.bso("myid").deleted);
server.stop(run_next_test);
});
add_test(function test_missing_collection_404() {
_("Ensure a missing collection returns a 404.");
let server = new StorageServer();
server.registerUser("123", "password");
server.startSynchronous(PORT);
let request = localRequest("/2.0/123/storage/none", "123", "password");
let error = doGetRequest(request);
do_check_eq(error, null);
do_check_eq(request.response.status, 404);
do_check_false("content-type" in request.response.headers);
server.stop(run_next_test);
});
add_test(function test_get_storage_405() {
_("Ensure that a GET on /storage results in a 405.");
let server = new StorageServer();
server.registerUser("123", "password");
server.startSynchronous(PORT);
let request = localRequest("/2.0/123/storage", "123", "password");
let error = doGetRequest(request);
do_check_eq(error, null);
do_check_eq(request.response.status, 405);
do_check_eq(request.response.headers["allow"], "DELETE");
server.stop(run_next_test);
});
add_test(function test_delete_storage() {
_("Ensure that deleting all of storage works.");
let server = new StorageServer();
server.registerUser("123", "password");
server.createContents("123", {
foo: {a: {foo: "bar"}, b: {bar: "foo"}},
baz: {c: {bob: "law"}, blah: {law: "blog"}}
});
server.startSynchronous(PORT);
let request = localRequest("/2.0/123/storage", "123", "password");
let error = doDeleteRequest(request);
do_check_eq(error, null);
do_check_eq(request.response.status, 204);
do_check_attribute_count(server.users["123"].collections, 0);
server.stop(run_next_test);
});
add_test(function test_x_num_records() {
let server = new StorageServer();
server.registerUser("123", "password");
server.createContents("123", {
crypto: {foos: {foo: "bar"},
bars: {foo: "baz"}}
});
server.startSynchronous(PORT);
let bso = localRequest("/2.0/123/storage/crypto/foos");
bso.get(function (err) {
// BSO fetches don't have one.
do_check_false("x-num-records" in this.response.headers);
let col = localRequest("/2.0/123/storage/crypto");
col.get(function (err) {
// Collection fetches do.
do_check_eq(this.response.headers["x-num-records"], "2");
server.stop(run_next_test);
});
});
});

27
services/common/tests/unit/test_utils_encodeBase64URL.js Normal file
View File

@ -0,0 +1,27 @@
/* Any copyright is dedicated to the Public Domain.
http://creativecommons.org/publicdomain/zero/1.0/ */
Cu.import("resource://services-common/utils.js");
function run_test() {
run_next_test();
}
add_test(function test_simple() {
let expected = {
hello: "aGVsbG8=",
"<>?": "PD4_",
};
for (let [k, v] in Iterator(expected)) {
do_check_eq(CommonUtils.encodeBase64URL(k), v);
}
run_next_test();
});
add_test(function test_no_padding() {
do_check_eq(CommonUtils.encodeBase64URL("hello", false), "aGVsbG8");
run_next_test();
});

12
services/common/tests/unit/test_utils_json.js
View File

@ -11,7 +11,9 @@ function run_test() {
add_test(function test_roundtrip() {
_("Do a simple write of an array to json and read");
CommonUtils.jsonSave("foo", {}, ["v1", "v2"], ensureThrows(function() {
CommonUtils.jsonSave("foo", {}, ["v1", "v2"], ensureThrows(function(error) {
do_check_eq(error, null);
CommonUtils.jsonLoad("foo", {}, ensureThrows(function(val) {
let foo = val;
do_check_eq(typeof foo, "object");
@ -25,7 +27,9 @@ add_test(function test_roundtrip() {
add_test(function test_string() {
_("Try saving simple strings");
CommonUtils.jsonSave("str", {}, "hi", ensureThrows(function() {
CommonUtils.jsonSave("str", {}, "hi", ensureThrows(function(error) {
do_check_eq(error, null);
CommonUtils.jsonLoad("str", {}, ensureThrows(function(val) {
let str = val;
do_check_eq(typeof str, "string");
@ -39,7 +43,9 @@ add_test(function test_string() {
add_test(function test_number() {
_("Try saving a number");
CommonUtils.jsonSave("num", {}, 42, ensureThrows(function() {
CommonUtils.jsonSave("num", {}, 42, ensureThrows(function(error) {
do_check_eq(error, null);
CommonUtils.jsonLoad("num", {}, ensureThrows(function(val) {
let num = val;
do_check_eq(typeof num, "number");

6
services/common/tests/unit/xpcshell.ini
View File

@ -1,5 +1,5 @@
[DEFAULT]
head = head_global.js head_helpers.js
head = head_global.js head_helpers.js head_http.js aitcserver.js storageserver.js
tail =
# Test load modules first so syntax failures are caught early.
@ -7,12 +7,14 @@ tail =
[test_utils_atob.js]
[test_utils_encodeBase32.js]
[test_utils_encodeBase64URL.js]
[test_utils_json.js]
[test_utils_makeURI.js]
[test_utils_namedTimer.js]
[test_utils_stackTrace.js]
[test_utils_utf8.js]
[test_aitc_server.js]
[test_async_chain.js]
[test_async_querySpinningly.js]
[test_log4moz.js]
@ -21,3 +23,5 @@ tail =
[test_restrequest.js]
[test_tokenauthenticatedrequest.js]
[test_tokenserverclient.js]
[test_storage_server.js]

90
services/common/utils.js
View File

@ -59,11 +59,23 @@ let CommonUtils = {
/**
* Encode byte string as base64URL (RFC 4648).
*
* @param bytes
* (string) Raw byte string to encode.
* @param pad
* (bool) Whether to include padding characters (=). Defaults
* to true for historical reasons.
*/
encodeBase64URL: function encodeBase64URL(bytes) {
return btoa(bytes).replace("+", "-", "g").replace("/", "_", "g");
encodeBase64URL: function encodeBase64URL(bytes, pad=true) {
let s = btoa(bytes).replace("+", "-", "g").replace("/", "_", "g");
if (!pad) {
s = s.replace("=", "");
}
return s;
},
/**
* Create a nsIURI instance from a string.
*/
@ -94,6 +106,20 @@ let CommonUtils = {
Services.tm.currentThread.dispatch(callback, Ci.nsIThread.DISPATCH_NORMAL);
},
/**
* Spin the event loop and return once the next tick is executed.
*
* This is an evil function and should not be used in production code. It
* exists in this module for ease-of-use.
*/
waitForNextTick: function waitForNextTick() {
let cb = Async.makeSyncCallback();
this.nextTick(cb);
Async.waitForSyncCallback(cb);
return;
},
/**
* Return a timer that is scheduled to call the callback after waiting the
* provided time or as soon as possible. The timer will be set as a property
@ -364,6 +390,9 @@ let CommonUtils = {
* function, it'll be used as the object to make a json string.
* @param callback
* Function called when the write has been performed. Optional.
* The first argument will be a Components.results error
* constant on error or null if no error was encountered (and
* the file saved successfully).
*/
jsonSave: function jsonSave(filePath, that, obj, callback) {
let path = filePath + ".json";
@ -379,10 +408,63 @@ let CommonUtils = {
let is = this._utf8Converter.convertToInputStream(out);
NetUtil.asyncCopy(is, fos, function (result) {
if (typeof callback == "function") {
callback.call(that);
let error = (result == Cr.NS_OK) ? null : result;
callback.call(that, error);
}
});
},
/**
* Ensure that the specified value is defined in integer milliseconds since
* UNIX epoch.
*
* This throws an error if the value is not an integer, is negative, or looks
* like seconds, not milliseconds.
*
* If the value is null or 0, no exception is raised.
*
* @param value
* Value to validate.
*/
ensureMillisecondsTimestamp: function ensureMillisecondsTimestamp(value) {
if (!value) {
return;
}
if (value < 0) {
throw new Error("Timestamp value is negative: " + value);
}
// Catch what looks like seconds, not milliseconds.
if (value < 10000000000) {
throw new Error("Timestamp appears to be in seconds: " + value);
}
if (Math.floor(value) != Math.ceil(value)) {
throw new Error("Timestamp value is not an integer: " + value);
}
},
/**
* Read bytes from an nsIInputStream into a string.
*
* @param stream
* (nsIInputStream) Stream to read from.
* @param count
* (number) Integer number of bytes to read. If not defined, or
* 0, all available input is read.
*/
readBytesFromInputStream: function readBytesFromInputStream(stream, count) {
let BinaryInputStream = Components.Constructor(
"@mozilla.org/binaryinputstream;1",
"nsIBinaryInputStream",
"setInputStream");
if (!count) {
count = stream.available();
}
return new BinaryInputStream(stream).readBytes(count);
},
};
XPCOMUtils.defineLazyGetter(CommonUtils, "_utf8Converter", function() {

3
testing/xpcshell/xpcshell.ini
View File

@ -73,11 +73,12 @@ skip-if = os == "android"
[include:content/base/test/unit/xpcshell.ini]
[include:content/test/unit/xpcshell.ini]
[include:toolkit/components/url-classifier/tests/unit/xpcshell.ini]
[include:services/aitc/tests/unit/xpcshell.ini]
[include:services/common/tests/unit/xpcshell.ini]
[include:services/crypto/tests/unit/xpcshell.ini]
[include:services/crypto/components/tests/unit/xpcshell.ini]
[include:services/sync/tests/unit/xpcshell.ini]
# Bug 676978: tests hang on Android
# Bug 676978: tests hang on Android
skip-if = os == "android"
[include:browser/components/dirprovider/tests/unit/xpcshell.ini]
[include:browser/components/downloads/test/unit/xpcshell.ini]

10
uriloader/prefetch/OfflineCacheUpdateChild.cpp
View File

@ -177,8 +177,9 @@ OfflineCacheUpdateChild::AssociateDocument(nsIDOMDocument *aDocument,
NS_IMETHODIMP
OfflineCacheUpdateChild::Init(nsIURI *aManifestURI,
nsIURI *aDocumentURI,
nsIDOMDocument *aDocument)
nsIURI *aDocumentURI,
nsIDOMDocument *aDocument,
nsILocalFile *aCustomProfileDir)
{
nsresult rv;
@ -188,6 +189,11 @@ OfflineCacheUpdateChild::Init(nsIURI *aManifestURI,
if (!service)
return NS_ERROR_FAILURE;
if (aCustomProfileDir) {
NS_ERROR("Custom Offline Cache Update not supported on child process");
return NS_ERROR_NOT_IMPLEMENTED;
}
LOG(("OfflineCacheUpdateChild::Init [%p]", this));
// Only http and https applications are supported.

5
uriloader/prefetch/OfflineCacheUpdateGlue.cpp
View File

@ -91,7 +91,8 @@ OfflineCacheUpdateGlue::Schedule()
NS_IMETHODIMP
OfflineCacheUpdateGlue::Init(nsIURI *aManifestURI,
nsIURI *aDocumentURI,
nsIDOMDocument *aDocument)
nsIDOMDocument *aDocument,
nsILocalFile *aCustomProfileDir)
{
if (!EnsureUpdate())
return NS_ERROR_NULL_POINTER;
@ -101,7 +102,7 @@ OfflineCacheUpdateGlue::Init(nsIURI *aManifestURI,
if (aDocument)
SetDocument(aDocument);
return mUpdate->Init(aManifestURI, aDocumentURI, nsnull);
return mUpdate->Init(aManifestURI, aDocumentURI, nsnull, aCustomProfileDir);
}
void

7
uriloader/prefetch/OfflineCacheUpdateGlue.h
View File

@ -35,8 +35,8 @@ namespace docshell {
NS_SCRIPTABLE NS_IMETHOD GetByteProgress(PRUint64 * _result) { return !_to ? NS_ERROR_NULL_POINTER : _to->GetByteProgress(_result); }
class OfflineCacheUpdateGlue : public nsSupportsWeakReference
, public nsIOfflineCacheUpdate
, public nsIOfflineCacheUpdateObserver
, public nsIOfflineCacheUpdate
, public nsIOfflineCacheUpdateObserver
{
public:
NS_DECL_ISUPPORTS
@ -49,7 +49,8 @@ public:
NS_SCRIPTABLE NS_IMETHOD Schedule(void);
NS_SCRIPTABLE NS_IMETHOD Init(nsIURI *aManifestURI,
nsIURI *aDocumentURI,
nsIDOMDocument *aDocument);
nsIDOMDocument *aDocument,
nsILocalFile *aCustomProfileDir);
NS_DECL_NSIOFFLINECACHEUPDATEOBSERVER

2
uriloader/prefetch/OfflineCacheUpdateParent.cpp
View File

@ -83,7 +83,7 @@ OfflineCacheUpdateParent::Schedule(const URI& aManifestURI,
nsresult rv;
// Leave aDocument argument null. Only glues and children keep
// document instances.
rv = update->Init(manifestURI, documentURI, nsnull);
rv = update->Init(manifestURI, documentURI, nsnull, nsnull);
NS_ENSURE_SUCCESS(rv, rv);
rv = update->Schedule();

15
uriloader/prefetch/nsIOfflineCacheUpdate.idl
View File

@ -14,6 +14,7 @@ interface nsIOfflineCacheUpdate;
interface nsIPrincipal;
interface nsIPrefBranch;
interface nsIApplicationCache;
interface nsILocalFile;
[scriptable, uuid(47360d57-8ef4-4a5d-8865-1a27a739ad1a)]
interface nsIOfflineCacheUpdateObserver : nsISupports {
@ -105,7 +106,8 @@ interface nsIOfflineCacheUpdate : nsISupports {
* @param aDocumentURI
* The page that is requesting the update.
*/
void init(in nsIURI aManifestURI, in nsIURI aDocumentURI, in nsIDOMDocument aDocument);
void init(in nsIURI aManifestURI, in nsIURI aDocumentURI, in nsIDOMDocument aDocument,
[optional] in nsILocalFile aCustomProfileDir);
/**
* Initialize the update for partial processing.
@ -162,7 +164,7 @@ interface nsIOfflineCacheUpdate : nsISupports {
readonly attribute PRUint64 byteProgress;
};
[scriptable, uuid(6fd2030f-7b00-4102-a0e3-d73078821eb1)]
[scriptable, uuid(dc5de18c-197c-41d2-9584-dd7ac7494611)]
interface nsIOfflineCacheUpdateService : nsISupports {
/**
* Constants for the offline-app permission.
@ -192,6 +194,15 @@ interface nsIOfflineCacheUpdateService : nsISupports {
in nsIURI aDocumentURI,
in nsIDOMWindow aWindow);
/**
* Schedule a cache update for a given offline manifest and let the data
* be stored to a custom profile directory. There is no coalescing of
* manifests by manifest URL.
*/
nsIOfflineCacheUpdate scheduleCustomProfileUpdate(in nsIURI aManifestURI,
in nsIURI aDocumentURI,
in nsILocalFile aProfileDir);
/**
* Schedule a cache update for a manifest when the document finishes
* loading.

243
uriloader/prefetch/nsOfflineCacheUpdate.cpp
View File

@ -45,6 +45,11 @@ using namespace mozilla;
static const PRUint32 kRescheduleLimit = 3;
// Max number of retries for every entry of pinned app.
static const PRUint32 kPinnedEntryRetriesLimit = 3;
// Maximum number of parallel items loads
static const PRUint32 kParallelLoadLimit = 15;
// Quota for offline apps when preloading
static const PRInt32 kCustomProfileQuota = 512000;
#if defined(PR_LOGGING)
//
@ -280,18 +285,18 @@ NS_IMPL_ISUPPORTS6(nsOfflineCacheUpdateItem,
// nsOfflineCacheUpdateItem <public>
//-----------------------------------------------------------------------------
nsOfflineCacheUpdateItem::nsOfflineCacheUpdateItem(nsOfflineCacheUpdate *aUpdate,
nsIURI *aURI,
nsOfflineCacheUpdateItem::nsOfflineCacheUpdateItem(nsIURI *aURI,
nsIURI *aReferrerURI,
nsIApplicationCache *aApplicationCache,
nsIApplicationCache *aPreviousApplicationCache,
const nsACString &aClientID,
PRUint32 type)
: mURI(aURI)
, mReferrerURI(aReferrerURI)
, mApplicationCache(aApplicationCache)
, mPreviousApplicationCache(aPreviousApplicationCache)
, mClientID(aClientID)
, mItemType(type)
, mUpdate(aUpdate)
, mChannel(nsnull)
, mState(nsIDOMLoadStatus::UNINITIALIZED)
, mBytesRead(0)
@ -303,7 +308,7 @@ nsOfflineCacheUpdateItem::~nsOfflineCacheUpdateItem()
}
nsresult
nsOfflineCacheUpdateItem::OpenChannel()
nsOfflineCacheUpdateItem::OpenChannel(nsOfflineCacheUpdate *aUpdate)
{
#if defined(PR_LOGGING)
if (LOG_ENABLED()) {
@ -350,6 +355,13 @@ nsOfflineCacheUpdateItem::OpenChannel()
rv = cachingChannel->SetCacheForOfflineUse(true);
NS_ENSURE_SUCCESS(rv, rv);
nsCOMPtr<nsILocalFile> cacheDirectory;
rv = mApplicationCache->GetCacheDirectory(getter_AddRefs(cacheDirectory));
NS_ENSURE_SUCCESS(rv, rv);
rv = cachingChannel->SetProfileDirectory(cacheDirectory);
NS_ENSURE_SUCCESS(rv, rv);
if (!mClientID.IsEmpty()) {
rv = cachingChannel->SetOfflineCacheClientID(mClientID);
NS_ENSURE_SUCCESS(rv, rv);
@ -359,6 +371,8 @@ nsOfflineCacheUpdateItem::OpenChannel()
rv = mChannel->AsyncOpen(this, nsnull);
NS_ENSURE_SUCCESS(rv, rv);
mUpdate = aUpdate;
mState = nsIDOMLoadStatus::REQUESTED;
return NS_OK;
@ -415,8 +429,6 @@ nsOfflineCacheUpdateItem::OnStopRequest(nsIRequest *aRequest,
{
LOG(("done fetching offline item [status=%x]\n", aStatus));
mState = nsIDOMLoadStatus::LOADED;
if (mBytesRead == 0 && aStatus == NS_OK) {
// we didn't need to read (because LOAD_ONLY_IF_MODIFIED was
// specified), but the object should report loadedSize as if it
@ -439,7 +451,17 @@ nsOfflineCacheUpdateItem::OnStopRequest(nsIRequest *aRequest,
NS_IMETHODIMP
nsOfflineCacheUpdateItem::Run()
{
mUpdate->LoadCompleted();
// Set mState to LOADED here rather than in OnStopRequest to prevent
// race condition when checking state of all mItems in ProcessNextURI().
// If state would have been set in OnStopRequest we could mistakenly
// take this item as already finished and finish the update process too
// early when ProcessNextURI() would get called between OnStopRequest()
// and Run() of this item. Finish() would then have been called twice.
mState = nsIDOMLoadStatus::LOADED;
nsRefPtr<nsOfflineCacheUpdate> update;
update.swap(mUpdate);
update->LoadCompleted(this);
return NS_OK;
}
@ -487,10 +509,18 @@ nsOfflineCacheUpdateItem::AsyncOnChannelRedirect(nsIChannel *aOldChannel,
if (newCachingChannel) {
rv = newCachingChannel->SetCacheForOfflineUse(true);
NS_ENSURE_SUCCESS(rv, rv);
if (!mClientID.IsEmpty()) {
rv = newCachingChannel->SetOfflineCacheClientID(mClientID);
NS_ENSURE_SUCCESS(rv, rv);
}
nsCOMPtr<nsILocalFile> cacheDirectory;
rv = mApplicationCache->GetCacheDirectory(getter_AddRefs(cacheDirectory));
NS_ENSURE_SUCCESS(rv, rv);
rv = newCachingChannel->SetProfileDirectory(cacheDirectory);
NS_ENSURE_SUCCESS(rv, rv);
}
nsCAutoString oldScheme;
@ -599,6 +629,25 @@ nsOfflineCacheUpdateItem::GetRequestSucceeded(bool * succeeded)
return NS_OK;
}
bool
nsOfflineCacheUpdateItem::IsScheduled()
{
return mState == nsIDOMLoadStatus::UNINITIALIZED;
}
bool
nsOfflineCacheUpdateItem::IsInProgress()
{
return mState == nsIDOMLoadStatus::REQUESTED ||
mState == nsIDOMLoadStatus::RECEIVING;
}
bool
nsOfflineCacheUpdateItem::IsCompleted()
{
return mState == nsIDOMLoadStatus::LOADED;
}
NS_IMETHODIMP
nsOfflineCacheUpdateItem::GetStatus(PRUint16 *aStatus)
{
@ -631,13 +680,13 @@ nsOfflineCacheUpdateItem::GetStatus(PRUint16 *aStatus)
// nsOfflineManifestItem <public>
//-----------------------------------------------------------------------------
nsOfflineManifestItem::nsOfflineManifestItem(nsOfflineCacheUpdate *aUpdate,
nsIURI *aURI,
nsOfflineManifestItem::nsOfflineManifestItem(nsIURI *aURI,
nsIURI *aReferrerURI,
nsIApplicationCache *aApplicationCache,
nsIApplicationCache *aPreviousApplicationCache,
const nsACString &aClientID)
: nsOfflineCacheUpdateItem(aUpdate, aURI, aReferrerURI,
aPreviousApplicationCache, aClientID,
: nsOfflineCacheUpdateItem(aURI, aReferrerURI,
aApplicationCache, aPreviousApplicationCache, aClientID,
nsIApplicationCache::ITEM_MANIFEST)
, mParserState(PARSE_INIT)
, mNeedsUpdate(true)
@ -709,6 +758,7 @@ nsOfflineManifestItem::ReadManifest(nsIInputStream *aInputStream,
if (NS_FAILED(rv)) {
LOG(("HandleManifestLine failed with 0x%08x", rv));
*aBytesConsumed = 0; // Avoid assertion failure in stream tee
return NS_ERROR_ABORT;
}
@ -1096,9 +1146,10 @@ nsOfflineManifestItem::OnStopRequest(nsIRequest *aRequest,
// nsOfflineCacheUpdate::nsISupports
//-----------------------------------------------------------------------------
NS_IMPL_ISUPPORTS2(nsOfflineCacheUpdate,
NS_IMPL_ISUPPORTS3(nsOfflineCacheUpdate,
nsIOfflineCacheUpdateObserver,
nsIOfflineCacheUpdate)
nsIOfflineCacheUpdate,
nsIRunnable)
//-----------------------------------------------------------------------------
// nsOfflineCacheUpdate <public>
@ -1111,7 +1162,7 @@ nsOfflineCacheUpdate::nsOfflineCacheUpdate()
, mPartialUpdate(false)
, mSucceeded(true)
, mObsolete(false)
, mCurrentItem(-1)
, mItemsInProgress(0)
, mRescheduleCount(0)
, mPinnedEntryRetriesCount(0)
, mPinned(false)
@ -1142,7 +1193,8 @@ nsOfflineCacheUpdate::GetCacheKey(nsIURI *aURI, nsACString &aKey)
nsresult
nsOfflineCacheUpdate::Init(nsIURI *aManifestURI,
nsIURI *aDocumentURI,
nsIDOMDocument *aDocument)
nsIDOMDocument *aDocument,
nsILocalFile *aCustomProfileDir)
{
nsresult rv;
@ -1184,13 +1236,32 @@ nsOfflineCacheUpdate::Init(nsIURI *aManifestURI,
do_GetService(NS_APPLICATIONCACHESERVICE_CONTRACTID, &rv);
NS_ENSURE_SUCCESS(rv, rv);
rv = cacheService->GetActiveCache(manifestSpec,
getter_AddRefs(mPreviousApplicationCache));
NS_ENSURE_SUCCESS(rv, rv);
if (aCustomProfileDir) {
// Create only a new offline application cache in the custom profile
// This is a preload of a new cache.
rv = cacheService->CreateApplicationCache(manifestSpec,
getter_AddRefs(mApplicationCache));
NS_ENSURE_SUCCESS(rv, rv);
// XXX Custom updates don't support "updating" of an existing cache
// in the custom profile at the moment. This support can be, though,
// simply added as well when needed.
mPreviousApplicationCache = nsnull;
rv = cacheService->CreateCustomApplicationCache(manifestSpec,
aCustomProfileDir,
kCustomProfileQuota,
getter_AddRefs(mApplicationCache));
NS_ENSURE_SUCCESS(rv, rv);
mCustomProfileDir = aCustomProfileDir;
}
else {
rv = cacheService->GetActiveCache(manifestSpec,
getter_AddRefs(mPreviousApplicationCache));
NS_ENSURE_SUCCESS(rv, rv);
rv = cacheService->CreateApplicationCache(manifestSpec,
getter_AddRefs(mApplicationCache));
NS_ENSURE_SUCCESS(rv, rv);
}
rv = mApplicationCache->GetClientID(mClientID);
NS_ENSURE_SUCCESS(rv, rv);
@ -1317,15 +1388,20 @@ nsOfflineCacheUpdate::HandleManifest(bool *aDoUpdate)
}
void
nsOfflineCacheUpdate::LoadCompleted()
nsOfflineCacheUpdate::LoadCompleted(nsOfflineCacheUpdateItem *aItem)
{
nsresult rv;
LOG(("nsOfflineCacheUpdate::LoadCompleted [%p]", this));
if (mState == STATE_FINISHED) {
LOG((" after completion, ignoring"));
return;
}
// Keep the object alive through a Finish() call.
nsCOMPtr<nsIOfflineCacheUpdate> kungFuDeathGrip(this);
LOG(("nsOfflineCacheUpdate::LoadCompleted [%p]", this));
if (mState == STATE_CANCELLED) {
Finish();
return;
@ -1336,6 +1412,8 @@ nsOfflineCacheUpdate::LoadCompleted()
NS_ASSERTION(mManifestItem,
"Must have a manifest item in STATE_CHECKING.");
NS_ASSERTION(mManifestItem == aItem,
"Unexpected aItem in nsOfflineCacheUpdate::LoadCompleted");
// A 404 or 410 is interpreted as an intentional removal of
// the manifest file, rather than a transient server error.
@ -1403,21 +1481,21 @@ nsOfflineCacheUpdate::LoadCompleted()
}
// Normal load finished.
nsRefPtr<nsOfflineCacheUpdateItem> item = mItems[mCurrentItem];
if (mItemsInProgress) // Just to be safe here!
--mItemsInProgress;
bool succeeded;
rv = item->GetRequestSucceeded(&succeeded);
rv = aItem->GetRequestSucceeded(&succeeded);
if (mPinned) {
if (mPinned && NS_SUCCEEDED(rv) && succeeded) {
PRUint32 dummy_cache_type;
rv = mApplicationCache->GetTypes(item->mCacheKey, &dummy_cache_type);
rv = mApplicationCache->GetTypes(aItem->mCacheKey, &dummy_cache_type);
bool item_doomed = NS_FAILED(rv); // can not find it? -> doomed
if (item_doomed &&
mPinnedEntryRetriesCount < kPinnedEntryRetriesLimit &&
(item->mItemType & (nsIApplicationCache::ITEM_EXPLICIT |
nsIApplicationCache::ITEM_FALLBACK))) {
(aItem->mItemType & (nsIApplicationCache::ITEM_EXPLICIT |
nsIApplicationCache::ITEM_FALLBACK))) {
rv = EvictOneNonPinned();
if (NS_FAILED(rv)) {
mSucceeded = false;
@ -1426,7 +1504,9 @@ nsOfflineCacheUpdate::LoadCompleted()
return;
}
rv = item->Cancel();
// This reverts the item state to UNINITIALIZED that makes it to
// be scheduled for download again.
rv = aItem->Cancel();
if (NS_FAILED(rv)) {
mSucceeded = false;
NotifyState(nsIOfflineCacheUpdateObserver::STATE_ERROR);
@ -1435,26 +1515,29 @@ nsOfflineCacheUpdate::LoadCompleted()
}
mPinnedEntryRetriesCount++;
// Retry current item, so mCurrentItem is not advanced.
// Retry this item.
ProcessNextURI();
return;
}
}
// Advance to next item.
mCurrentItem++;
// According to parallelism this may imply more pinned retries count,
// but that is not critical, since at one moment the algoritm will
// stop anyway. Also, this code may soon be completely removed
// after we have a separate storage for pinned apps.
mPinnedEntryRetriesCount = 0;
// Check for failures. 3XX, 4XX and 5XX errors on items explicitly
// listed in the manifest will cause the update to fail.
if (NS_FAILED(rv) || !succeeded) {
if (item->mItemType &
if (aItem->mItemType &
(nsIApplicationCache::ITEM_EXPLICIT |
nsIApplicationCache::ITEM_FALLBACK)) {
mSucceeded = false;
}
} else {
rv = mApplicationCache->MarkEntry(item->mCacheKey, item->mItemType);
rv = mApplicationCache->MarkEntry(aItem->mCacheKey, aItem->mItemType);
if (NS_FAILED(rv)) {
mSucceeded = false;
}
@ -1505,7 +1588,7 @@ nsOfflineCacheUpdate::ManifestCheckCompleted(nsresult aStatus,
new nsOfflineCacheUpdate();
// Leave aDocument argument null. Only glues and children keep
// document instances.
newUpdate->Init(mManifestURI, mDocumentURI, nsnull);
newUpdate->Init(mManifestURI, mDocumentURI, nsnull, mCustomProfileDir);
// In a rare case the manifest will not be modified on the next refetch
// transfer all master document URIs to the new update to ensure that
@ -1531,7 +1614,7 @@ nsOfflineCacheUpdate::Begin()
// Keep the object alive through a ProcessNextURI()/Finish() call.
nsCOMPtr<nsIOfflineCacheUpdate> kungFuDeathGrip(this);
mCurrentItem = 0;
mItemsInProgress = 0;
if (mPartialUpdate) {
mState = STATE_DOWNLOADING;
@ -1543,8 +1626,9 @@ nsOfflineCacheUpdate::Begin()
// Start checking the manifest.
nsCOMPtr<nsIURI> uri;
mManifestItem = new nsOfflineManifestItem(this, mManifestURI,
mManifestItem = new nsOfflineManifestItem(mManifestURI,
mDocumentURI,
mApplicationCache,
mPreviousApplicationCache,
mClientID);
if (!mManifestItem) {
@ -1555,9 +1639,9 @@ nsOfflineCacheUpdate::Begin()
mByteProgress = 0;
NotifyState(nsIOfflineCacheUpdateObserver::STATE_CHECKING);
nsresult rv = mManifestItem->OpenChannel();
nsresult rv = mManifestItem->OpenChannel(this);
if (NS_FAILED(rv)) {
LoadCompleted();
LoadCompleted(mManifestItem);
}
return NS_OK;
@ -1571,10 +1655,12 @@ nsOfflineCacheUpdate::Cancel()
mState = STATE_CANCELLED;
mSucceeded = false;
if (mCurrentItem >= 0 &&
mCurrentItem < static_cast<PRInt32>(mItems.Length())) {
// Load might be running
mItems[mCurrentItem]->Cancel();
// Cancel all running downloads
for (PRUint32 i = 0; i < mItems.Length(); ++i) {
nsOfflineCacheUpdateItem * item = mItems[i];
if (item->IsInProgress())
item->Cancel();
}
return NS_OK;
@ -1634,13 +1720,29 @@ nsOfflineCacheUpdate::ProcessNextURI()
// Keep the object alive through a Finish() call.
nsCOMPtr<nsIOfflineCacheUpdate> kungFuDeathGrip(this);
LOG(("nsOfflineCacheUpdate::ProcessNextURI [%p, current=%d, numItems=%d]",
this, mCurrentItem, mItems.Length()));
LOG(("nsOfflineCacheUpdate::ProcessNextURI [%p, inprogress=%d, numItems=%d]",
this, mItemsInProgress, mItems.Length()));
NS_ASSERTION(mState == STATE_DOWNLOADING,
"ProcessNextURI should only be called from the DOWNLOADING state");
if (mCurrentItem >= static_cast<PRInt32>(mItems.Length())) {
nsOfflineCacheUpdateItem * runItem = nsnull;
PRUint32 completedItems = 0;
for (PRUint32 i = 0; i < mItems.Length(); ++i) {
nsOfflineCacheUpdateItem * item = mItems[i];
if (item->IsScheduled()) {
runItem = item;
break;
}
if (item->IsCompleted())
++completedItems;
}
if (completedItems == mItems.Length()) {
LOG(("nsOfflineCacheUpdate::ProcessNextURI [%p]: all items loaded", this));
if (mPartialUpdate) {
return Finish();
} else {
@ -1660,23 +1762,38 @@ nsOfflineCacheUpdate::ProcessNextURI()
}
}
if (!runItem) {
LOG(("nsOfflineCacheUpdate::ProcessNextURI [%p]:"
" No more items to include in parallel load", this));
return NS_OK;
}
#if defined(PR_LOGGING)
if (LOG_ENABLED()) {
nsCAutoString spec;
mItems[mCurrentItem]->mURI->GetSpec(spec);
runItem->mURI->GetSpec(spec);
LOG(("%p: Opening channel for %s", this, spec.get()));
}
#endif
++mItemsInProgress;
NotifyState(nsIOfflineCacheUpdateObserver::STATE_ITEMSTARTED);
nsresult rv = mItems[mCurrentItem]->OpenChannel();
nsresult rv = runItem->OpenChannel(this);
if (NS_FAILED(rv)) {
LoadCompleted();
LoadCompleted(runItem);
return rv;
}
return NS_OK;
if (mItemsInProgress >= kParallelLoadLimit) {
LOG(("nsOfflineCacheUpdate::ProcessNextURI [%p]:"
" At parallel load limit", this));
return NS_OK;
}
// This calls this method again via a post triggering
// a parallel item load
return NS_DispatchToCurrentThread(this);
}
nsresult
@ -1836,7 +1953,7 @@ nsOfflineCacheUpdate::FinishNoNotify()
mApplicationCache->GetGroupID(groupID);
appCacheService->DeactivateGroup(groupID);
}
}
}
if (!mSucceeded) {
// Update was not merged, mark all the loads as failures
@ -2016,8 +2133,11 @@ nsOfflineCacheUpdate::AddURI(nsIURI *aURI, PRUint32 aType)
}
nsRefPtr<nsOfflineCacheUpdateItem> item =
new nsOfflineCacheUpdateItem(this, aURI, mDocumentURI,
mPreviousApplicationCache, mClientID,
new nsOfflineCacheUpdateItem(aURI,
mDocumentURI,
mApplicationCache,
mPreviousApplicationCache,
mClientID,
aType);
if (!item) return NS_ERROR_OUT_OF_MEMORY;
@ -2146,3 +2266,14 @@ nsOfflineCacheUpdate::ApplicationCacheAvailable(nsIApplicationCache *application
{
return AssociateDocuments(applicationCache);
}
//-----------------------------------------------------------------------------
// nsOfflineCacheUpdate::nsIRunable
//-----------------------------------------------------------------------------
NS_IMETHODIMP
nsOfflineCacheUpdate::Run()
{
ProcessNextURI();
return NS_OK;
}

25
uriloader/prefetch/nsOfflineCacheUpdate.h
View File

@ -52,9 +52,9 @@ public:
NS_DECL_NSIINTERFACEREQUESTOR
NS_DECL_NSICHANNELEVENTSINK
nsOfflineCacheUpdateItem(nsOfflineCacheUpdate *aUpdate,
nsIURI *aURI,
nsOfflineCacheUpdateItem(nsIURI *aURI,
nsIURI *aReferrerURI,
nsIApplicationCache *aApplicationCache,
nsIApplicationCache *aPreviousApplicationCache,
const nsACString &aClientID,
PRUint32 aType);
@ -62,17 +62,22 @@ public:
nsCOMPtr<nsIURI> mURI;
nsCOMPtr<nsIURI> mReferrerURI;
nsCOMPtr<nsIApplicationCache> mApplicationCache;
nsCOMPtr<nsIApplicationCache> mPreviousApplicationCache;
nsCString mClientID;
nsCString mCacheKey;
PRUint32 mItemType;
nsresult OpenChannel();
nsresult OpenChannel(nsOfflineCacheUpdate *aUpdate);
nsresult Cancel();
nsresult GetRequestSucceeded(bool * succeeded);
bool IsInProgress();
bool IsScheduled();
bool IsCompleted();
private:
nsOfflineCacheUpdate* mUpdate;
nsRefPtr<nsOfflineCacheUpdate> mUpdate;
nsCOMPtr<nsIChannel> mChannel;
PRUint16 mState;
@ -87,9 +92,9 @@ public:
NS_DECL_NSISTREAMLISTENER
NS_DECL_NSIREQUESTOBSERVER
nsOfflineManifestItem(nsOfflineCacheUpdate *aUpdate,
nsIURI *aURI,
nsOfflineManifestItem(nsIURI *aURI,
nsIURI *aReferrerURI,
nsIApplicationCache *aApplicationCache,
nsIApplicationCache *aPreviousApplicationCache,
const nsACString &aClientID);
virtual ~nsOfflineManifestItem();
@ -179,12 +184,14 @@ public:
class nsOfflineCacheUpdate : public nsIOfflineCacheUpdate
, public nsIOfflineCacheUpdateObserver
, public nsIRunnable
, public nsOfflineCacheUpdateOwner
{
public:
NS_DECL_ISUPPORTS
NS_DECL_NSIOFFLINECACHEUPDATE
NS_DECL_NSIOFFLINECACHEUPDATEOBSERVER
NS_DECL_NSIRUNNABLE
nsOfflineCacheUpdate();
~nsOfflineCacheUpdate();
@ -196,7 +203,7 @@ public:
nsresult Begin();
nsresult Cancel();
void LoadCompleted();
void LoadCompleted(nsOfflineCacheUpdateItem *aItem);
void ManifestCheckCompleted(nsresult aStatus,
const nsCString &aManifestHash);
void StickDocument(nsIURI *aDocumentURI);
@ -250,6 +257,7 @@ private:
nsCString mUpdateDomain;
nsCOMPtr<nsIURI> mManifestURI;
nsCOMPtr<nsIURI> mDocumentURI;
nsCOMPtr<nsILocalFile> mCustomProfileDir;
nsCString mClientID;
nsCOMPtr<nsIApplicationCache> mApplicationCache;
@ -260,7 +268,7 @@ private:
nsRefPtr<nsOfflineManifestItem> mManifestItem;
/* Items being updated */
PRInt32 mCurrentItem;
PRUint32 mItemsInProgress;
nsTArray<nsRefPtr<nsOfflineCacheUpdateItem> > mItems;
/* Clients watching this update for changes */
@ -309,6 +317,7 @@ public:
nsIURI *aDocumentURI,
nsIDOMDocument *aDocument,
nsIDOMWindow* aWindow,
nsILocalFile* aCustomProfileDir,
nsIOfflineCacheUpdate **aUpdate);
virtual nsresult UpdateFinished(nsOfflineCacheUpdate *aUpdate);

19
uriloader/prefetch/nsOfflineCacheUpdateService.cpp
View File

@ -161,7 +161,7 @@ nsOfflineCachePendingUpdate::OnStateChange(nsIWebProgress* aWebProgress,
if (NS_SUCCEEDED(aStatus)) {
nsCOMPtr<nsIOfflineCacheUpdate> update;
mService->Schedule(mManifestURI, mDocumentURI,
updateDoc, window, getter_AddRefs(update));
updateDoc, window, nsnull, getter_AddRefs(update));
}
aWebProgress->RemoveProgressListener(this);
@ -436,6 +436,7 @@ nsOfflineCacheUpdateService::Schedule(nsIURI *aManifestURI,
nsIURI *aDocumentURI,
nsIDOMDocument *aDocument,
nsIDOMWindow* aWindow,
nsILocalFile* aCustomProfileDir,
nsIOfflineCacheUpdate **aUpdate)
{
nsCOMPtr<nsIOfflineCacheUpdate> update;
@ -448,7 +449,7 @@ nsOfflineCacheUpdateService::Schedule(nsIURI *aManifestURI,
nsresult rv;
rv = update->Init(aManifestURI, aDocumentURI, aDocument);
rv = update->Init(aManifestURI, aDocumentURI, aDocument, aCustomProfileDir);
NS_ENSURE_SUCCESS(rv, rv);
rv = update->Schedule();
@ -465,7 +466,19 @@ nsOfflineCacheUpdateService::ScheduleUpdate(nsIURI *aManifestURI,
nsIDOMWindow *aWindow,
nsIOfflineCacheUpdate **aUpdate)
{
return Schedule(aManifestURI, aDocumentURI, nsnull, aWindow, aUpdate);
return Schedule(aManifestURI, aDocumentURI, nsnull, aWindow, nsnull, aUpdate);
}
NS_IMETHODIMP
nsOfflineCacheUpdateService::ScheduleCustomProfileUpdate(nsIURI *aManifestURI,
nsIURI *aDocumentURI,
nsILocalFile *aProfileDir,
nsIOfflineCacheUpdate **aUpdate)
{
// The profile directory is mandatory
NS_ENSURE_ARG(aProfileDir);
return Schedule(aManifestURI, aDocumentURI, nsnull, nsnull, aProfileDir, aUpdate);
}
//-----------------------------------------------------------------------------