Magic-Mirror/aria2
1
0
Fork 0
mirror of https://github.com/aria2/aria2.git synced 2025-02-16 04:07:49 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

2007-06-30 Tatsuhiro Tsujikawa <tujikawa at rednoah dot com>

Browse Source 
Made -S option work with metalink file and provided selective 
download
	to metalink.
	* src/MetalinkEntry.h, src/MetalinkEntry.cc
	(filename): Removed.
	(file): New variable.
	(size): Removed.
	(operator=): Updated.
	(getPath): New function.
	(getLength): New function.
	(toFileEntry): New function.
	* src/TorrentRequestInfo.h, src/TorrentRequestInfo.cc
	(execute): Use toStream.
	(showFileEntry): Removed.
	* src/MetalinkRequestInfo.h
	(targetFiles): New variable.
	(setTargetFiles): New variable.
	* src/MetalinkRequestInfo.cc
	(execute): Added the ability to print file information included 
in
	a metalink file.
	Added selective download mode to metalink.
	* src/main.cc
	(showUsage): Updated to denote that -S and --select-file options 
are
	applicable to metalink.
	* src/FileEntry.h
	(operator=): New function.
	(getBasename): New function.
	(getDirname): New function.
	* src/Util.h, src/Util.cc
	(toStream): New function.
	* src/Xml2MetalinkProcessor.cc: Updated.
	
	Made aria2 work with metalink with directory structure.
	* src/File.h, src/File.cc
	(getBasename): New function.
	(getDirname): New function.
	* src/RequestGroup.h, src/RequestGroup.cc
	(_topDir): New variable.
	(setTopDir): New function.
	(initSegmentMan): A directory structure is added to 
_segmentMan->dir.
	
	Rewrote HTTP header parsing with stringstream.
	* src/HttpConnection.h, src/HttpConnection.cc
	(HttpRequestEntry): New class.
	(headerBuf): Removed.
	(headerBufLength): Removed.
	(outstandingHttpRequests): Now its element type is
	HttpRequestEntryHandle.
	(findEndOfHeader): Removed.
	(receiveResponse): Rewritten.
	
	Updated doc for -j option to notice that it should be used with 
-i
	option.
	* src/main.cc (showUsage)

	Removed unused classes.
	* src/RequestInfo.h
	(FileInfo): Removed.
	(checksum): Removed.
	(fileInfo): Removed.
	(setChecksum): Removed.
	(getChecksum): Removed.
	(getFileInfo): Removed.

	Use ISO units.
	* src/ConsoleDownloadEngine.cc
	* src/TorrentConsoleDownloadEngine.cc
	* src/Util.cc (abbrevSize)
This commit is contained in:
Tatsuhiro Tsujikawa 2007-06-30 09:52:39 +00:00
parent 50b4cf67e0
commit e36a3de0f4
25 changed files with 360 additions and 233 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

70
ChangeLog
View File

@ -1,3 +1,73 @@
2007-06-30 Tatsuhiro Tsujikawa <tujikawa at rednoah dot com>
Made -S option work with metalink file and provided selective download
to metalink.
* src/MetalinkEntry.h, src/MetalinkEntry.cc
(filename): Removed.
(file): New variable.
(size): Removed.
(operator=): Updated.
(getPath): New function.
(getLength): New function.
(toFileEntry): New function.
* src/TorrentRequestInfo.h, src/TorrentRequestInfo.cc
(execute): Use toStream.
(showFileEntry): Removed.
* src/MetalinkRequestInfo.h
(targetFiles): New variable.
(setTargetFiles): New variable.
* src/MetalinkRequestInfo.cc
(execute): Added the ability to print file information included in
a metalink file.
Added selective download mode to metalink.
* src/main.cc
(showUsage): Updated to denote that -S and --select-file options are
applicable to metalink.
* src/FileEntry.h
(operator=): New function.
(getBasename): New function.
(getDirname): New function.
* src/Util.h, src/Util.cc
(toStream): New function.
* src/Xml2MetalinkProcessor.cc: Updated.
Made aria2 work with metalink with directory structure.
* src/File.h, src/File.cc
(getBasename): New function.
(getDirname): New function.
* src/RequestGroup.h, src/RequestGroup.cc
(_topDir): New variable.
(setTopDir): New function.
(initSegmentMan): A directory structure is added to _segmentMan->dir.
Rewrote HTTP header parsing with stringstream.
* src/HttpConnection.h, src/HttpConnection.cc
(HttpRequestEntry): New class.
(headerBuf): Removed.
(headerBufLength): Removed.
(outstandingHttpRequests): Now its element type is
HttpRequestEntryHandle.
(findEndOfHeader): Removed.
(receiveResponse): Rewritten.
Updated doc for -j option to notice that it should be used with -i
option.
* src/main.cc (showUsage)
Removed unused classes.
* src/RequestInfo.h
(FileInfo): Removed.
(checksum): Removed.
(fileInfo): Removed.
(setChecksum): Removed.
(getChecksum): Removed.
(getFileInfo): Removed.
Use ISO units.
* src/ConsoleDownloadEngine.cc
* src/TorrentConsoleDownloadEngine.cc
* src/Util.cc (abbrevSize)
2007-06-23 Tatsuhiro Tsujikawa <tujikawa at rednoah dot com>
Added the default listening ports to the help message.

10
src/ConsoleDownloadEngine.cc
View File

@ -64,8 +64,10 @@ void ConsoleDownloadEngine::sendStatistics(long long int currentSize, long long
cout << "["
<< "#" << firstRequestGroup->getGID() << " "
<< Util::abbrevSize(firstRequestGroup->getDownloadLength())
<< "B"
<< "/"
<< Util::abbrevSize(firstRequestGroup->getTotalLength());
<< Util::abbrevSize(firstRequestGroup->getTotalLength())
<< "B";
if(firstRequestGroup->getTotalLength() > 0) {
cout << "("
<< 100*firstRequestGroup->getDownloadLength()/firstRequestGroup->getTotalLength()
@ -81,7 +83,7 @@ void ConsoleDownloadEngine::sendStatistics(long long int currentSize, long long
}
cout << "]";
}
cout << "[" << fixed << setprecision(2) << speed/1024.0 << "KB/s" << "]";
cout << "[" << fixed << setprecision(2) << speed/1024.0 << "KiB/s" << "]";
{
FileAllocationEntryHandle entry = _fileAllocationMan->getCurrentFileAllocationEntry();
@ -89,8 +91,10 @@ void ConsoleDownloadEngine::sendStatistics(long long int currentSize, long long
cout << "[FileAlloc:"
<< "#" << entry->getRequestGroup()->getGID() << " "
<< Util::abbrevSize(entry->getCurrentLength())
<< "B"
<< "/"
<< Util::abbrevSize(entry->getTotalLength())
<< "B"
<< "("
<< 100*entry->getCurrentLength()/entry->getTotalLength()
<< "%)";
@ -108,8 +112,10 @@ void ConsoleDownloadEngine::sendStatistics(long long int currentSize, long long
cout << "[Checksum:"
<< "#" << entry->getRequestGroup()->getGID() << " "
<< Util::abbrevSize(entry->getCurrentLength())
<< "B"
<< "/"
<< Util::abbrevSize(entry->getTotalLength())
<< "B"
<< "("
<< 100*entry->getCurrentLength()/entry->getTotalLength()
<< "%)";

17
src/File.cc
View File

@ -37,6 +37,7 @@
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <libgen.h>
File::File(const string& name):name(name) {}
@ -120,3 +121,19 @@ mode_t File::mode()
}
return fstat.st_mode;
}
string File::getBasename() const
{
char* s = strdup(name.c_str());
string bname = basename(s);
free(s);
return bname;
}
string File::getDirname() const
{
char* s = strdup(name.c_str());
string dname = dirname(s);
free(s);
return dname;
}

4
src/File.h
View File

@ -86,6 +86,10 @@ public:
int64_t size();
mode_t mode();
string getBasename() const;
string getDirname() const;
};
#endif // _D_FILE_H_

31
src/FileEntry.h
View File

@ -36,24 +36,49 @@
#define _D_FILE_ENTRY_H_
#include "common.h"
#include "File.h"
class FileEntry {
private:
string path;
long long int length;
long long int offset;
int64_t length;
int64_t offset;
bool extracted;
bool requested;
public:
FileEntry():length(0), offset(0), extracted(false), requested(false) {}
FileEntry(const string& path, long long int length, long long int offset);
FileEntry& operator=(const FileEntry& entry)
{
if(this != &entry) {
path = entry.path;
length = entry.length;
offset = entry.offset;
extracted = entry.extracted;
requested = entry.requested;
}
return *this;
}
~FileEntry();
string getBasename() const
{
return File(path).getBasename();
}
string getDirname() const
{
return File(path).getDirname();
}
const string& getPath() const { return path; }
void setPath(const string& path) { this->path = path; }
long long int getLength() const { return length; }
int64_t getLength() const { return length; }
void setLength(long long int length) { this->length = length; }

102
src/HttpConnection.cc
View File

@ -34,6 +34,7 @@
/* copyright --> */
#include "HttpConnection.h"
#include "DlRetryEx.h"
#include "DlAbortEx.h"
#include "Util.h"
#include "Base64.h"
#include "message.h"
@ -44,9 +45,8 @@
HttpConnection::HttpConnection(int cuid,
const SocketHandle& socket,
const Option* op):
cuid(cuid), socket(socket), option(op), headerBufLength(0) {
logger = LogFactory::getInstance();
}
cuid(cuid), socket(socket), option(op), logger(LogFactory::getInstance())
{}
string HttpConnection::eraseConfidentialInfo(const string& request)
{
@ -70,7 +70,8 @@ void HttpConnection::sendRequest(const HttpRequestHandle& httpRequest)
string request = httpRequest->createRequest();
logger->info(MSG_SENDING_REQUEST, cuid, eraseConfidentialInfo(request).c_str());
socket->writeData(request.c_str(), request.size());
outstandingHttpRequests.push_back(httpRequest);
outstandingHttpRequests.push_back(new HttpRequestEntry(httpRequest,
new HttpHeaderProcessor()));
}
void HttpConnection::sendProxyRequest(const HttpRequestHandle& httpRequest)
@ -78,88 +79,41 @@ void HttpConnection::sendProxyRequest(const HttpRequestHandle& httpRequest)
string request = httpRequest->createProxyRequest();
logger->info(MSG_SENDING_REQUEST, cuid, eraseConfidentialInfo(request).c_str());
socket->writeData(request.c_str(), request.size());
outstandingHttpRequests.push_back(httpRequest);
}
int HttpConnection::findEndOfHeader(const char* buf, const char* substr, int bufLength) const {
const char* p = buf;
while(bufLength > p-buf && bufLength-(p-buf) >= (int)strlen(substr)) {
if(memcmp(p, substr, strlen(substr)) == 0) {
return p-buf;
}
p++;
}
return -1;
outstandingHttpRequests.push_back(new HttpRequestEntry(httpRequest,
new HttpHeaderProcessor()));
}
HttpResponseHandle HttpConnection::receiveResponse() {
//char buf[512];
string header;
int delimiterSwitch = 0;
char* delimiters[] = { "\r\n", "\n" };
int size = HEADERBUF_SIZE-headerBufLength;
if(size < 0) {
// TODO too large header
throw new DlRetryEx("too large header > 4096");
if(outstandingHttpRequests.size() == 0) {
throw new DlAbortEx("No HttpRequestEntry found.");
}
socket->peekData(headerBuf+headerBufLength, size);
HttpRequestEntryHandle entry = outstandingHttpRequests.front();
HttpHeaderProcessorHandle proc = entry->getHttpHeaderProcessor();
char buf[512];
int32_t size = sizeof(buf);
socket->peekData(buf, size);
if(size == 0) {
throw new DlRetryEx(EX_INVALID_RESPONSE);
}
//buf[size] = '\0';
int hlenTemp = headerBufLength+size;
//header += buf;
//string::size_type p;
int eohIndex;
if((eohIndex = findEndOfHeader(headerBuf, "\r\n\r\n", hlenTemp)) == -1 &&
(eohIndex = findEndOfHeader(headerBuf, "\n\n", hlenTemp)) == -1) {
socket->readData(headerBuf+headerBufLength, size);
headerBufLength += size;
} else {
if(headerBuf[eohIndex] == '\n') {
// for crapping non-standard HTTP server
delimiterSwitch = 1;
} else {
delimiterSwitch = 0;
}
headerBuf[eohIndex+strlen(delimiters[delimiterSwitch])*2] = '\0';
header = headerBuf;
size = eohIndex+strlen(delimiters[delimiterSwitch])*2-headerBufLength;
socket->readData(headerBuf+headerBufLength, size);
}
if(!Util::endsWith(header, "\r\n\r\n") && !Util::endsWith(header, "\n\n")) {
proc->update(buf, size);
if(!proc->eoh()) {
socket->readData(buf, size);
return 0;
}
int32_t putbackDataLength = proc->getPutBackDataLength();
size -= putbackDataLength;
socket->readData(buf, size);
// OK, we got all headers.
logger->info(MSG_RECEIVE_RESPONSE, cuid, header.c_str());
string::size_type p, np;
p = np = 0;
np = header.find(delimiters[delimiterSwitch], p);
if(np == string::npos) {
throw new DlRetryEx(EX_NO_STATUS_HEADER);
}
// check HTTP status value
if(header.size() <= 12) {
throw new DlRetryEx(EX_NO_STATUS_HEADER);
}
string status = header.substr(9, 3);
p = np+2;
HttpHeaderHandle httpHeader = new HttpHeader();
// retreive status name-value pairs, then push these into map
while((np = header.find(delimiters[delimiterSwitch], p)) != string::npos && np != p) {
string line = header.substr(p, np-p);
p = np+2;
pair<string, string> hp;
Util::split(hp, line, ':');
httpHeader->put(hp.first, hp.second);
}
logger->info(MSG_RECEIVE_RESPONSE, cuid, proc->getHeaderString().c_str());
pair<string, HttpHeaderHandle> httpStatusHeader = proc->getHttpStatusHeader();
HttpResponseHandle httpResponse = new HttpResponse();
httpResponse->setCuid(cuid);
httpResponse->setStatus(strtol(status.c_str(), 0, 10));
httpResponse->setHttpHeader(httpHeader);
httpResponse->setHttpRequest(outstandingHttpRequests.front());
httpResponse->setStatus(strtol(httpStatusHeader.first.c_str(), 0, 10));
httpResponse->setHttpHeader(httpStatusHeader.second);
httpResponse->setHttpRequest(entry->getHttpRequest());
outstandingHttpRequests.pop_front();

34
src/HttpConnection.h
View File

@ -43,10 +43,35 @@
#include "common.h"
#include "Logger.h"
#include "HttpResponse.h"
#include "HttpHeaderProcessor.h"
#include <netinet/in.h>
#include <string>
#define HEADERBUF_SIZE 4096
class HttpRequestEntry {
private:
HttpRequestHandle _httpRequest;
HttpHeaderProcessorHandle _proc;
public:
HttpRequestEntry(const HttpRequestHandle& httpRequest,
const HttpHeaderProcessorHandle& proc):
_httpRequest(httpRequest),
_proc(proc) {}
~HttpRequestEntry() {}
HttpRequestHandle getHttpRequest() const
{
return _httpRequest;
}
HttpHeaderProcessorHandle getHttpHeaderProcessor() const
{
return _proc;
}
};
typedef SharedHandle<HttpRequestEntry> HttpRequestEntryHandle;
typedef deque<HttpRequestEntryHandle> HttpRequestEntries;
class HttpConnection {
private:
@ -54,12 +79,9 @@ private:
SocketHandle socket;
const Option* option;
const Logger* logger;
char headerBuf[HEADERBUF_SIZE+1];
int headerBufLength;
HttpRequests outstandingHttpRequests;
HttpRequestEntries outstandingHttpRequests;
int findEndOfHeader(const char* buf, const char* substr, int bufLength) const;
string eraseConfidentialInfo(const string& request);
public:
HttpConnection(int cuid,
@ -96,7 +118,7 @@ public:
HttpRequestHandle getFirstHttpRequest() const
{
if(outstandingHttpRequests.size() > 0) {
return outstandingHttpRequests.front();
return outstandingHttpRequests.front()->getHttpRequest();
} else {
return 0;
}

5
src/Makefile.am
View File

@ -103,7 +103,9 @@ SRCS = Socket.h\
RequestGroupEntry.cc RequestGroupEntry.h\
Cookie.cc Cookie.h\
CookieParser.cc CookieParser.h\
CookieBoxFactory.cc CookieBoxFactory.h
CookieBoxFactory.cc CookieBoxFactory.h\
HttpHeaderProcessor.cc HttpHeaderProcessor.h\
FileEntry.cc FileEntry.h
# debug_new.cpp
if ENABLE_BITTORRENT
@ -133,7 +135,6 @@ SRCS += MetaEntry.h\
CopyDiskAdaptor.cc CopyDiskAdaptor.h\
DirectDiskAdaptor.cc DirectDiskAdaptor.h\
MultiDiskAdaptor.cc MultiDiskAdaptor.h\
FileEntry.cc FileEntry.h\
TrackerUpdateCommand.cc TrackerUpdateCommand.h\
ByteArrayDiskWriter.cc ByteArrayDiskWriter.h\
PeerChokeCommand.cc PeerChokeCommand.h\

70
src/Makefile.in
View File

@ -65,7 +65,6 @@ bin_PROGRAMS = aria2c$(EXEEXT)
@ENABLE_BITTORRENT_TRUE@ CopyDiskAdaptor.cc CopyDiskAdaptor.h\
@ENABLE_BITTORRENT_TRUE@ DirectDiskAdaptor.cc DirectDiskAdaptor.h\
@ENABLE_BITTORRENT_TRUE@ MultiDiskAdaptor.cc MultiDiskAdaptor.h\
@ENABLE_BITTORRENT_TRUE@ FileEntry.cc FileEntry.h\
@ENABLE_BITTORRENT_TRUE@ TrackerUpdateCommand.cc TrackerUpdateCommand.h\
@ENABLE_BITTORRENT_TRUE@ ByteArrayDiskWriter.cc ByteArrayDiskWriter.h\
@ENABLE_BITTORRENT_TRUE@ PeerChokeCommand.cc PeerChokeCommand.h\
@ -242,12 +241,13 @@ am__libaria2c_a_SOURCES_DIST = Socket.h SocketCore.cc SocketCore.h \
CheckIntegrityEntry.h CheckIntegrityMan.h ProgressAwareEntry.h \
RequestGroupEntry.cc RequestGroupEntry.h Cookie.cc Cookie.h \
CookieParser.cc CookieParser.h CookieBoxFactory.cc \
CookieBoxFactory.h MetaEntry.h Data.cc Data.h Dictionary.cc \
Dictionary.h List.cc List.h MetaFileUtil.cc MetaFileUtil.h \
MetaEntryVisitor.h ShaVisitor.cc ShaVisitor.h \
PeerConnection.cc PeerConnection.h PeerMessageUtil.cc \
PeerMessageUtil.h PeerAbstractCommand.cc PeerAbstractCommand.h \
PeerInitiateConnectionCommand.cc \
CookieBoxFactory.h HttpHeaderProcessor.cc \
HttpHeaderProcessor.h FileEntry.cc FileEntry.h MetaEntry.h \
Data.cc Data.h Dictionary.cc Dictionary.h List.cc List.h \
MetaFileUtil.cc MetaFileUtil.h MetaEntryVisitor.h \
ShaVisitor.cc ShaVisitor.h PeerConnection.cc PeerConnection.h \
PeerMessageUtil.cc PeerMessageUtil.h PeerAbstractCommand.cc \
PeerAbstractCommand.h PeerInitiateConnectionCommand.cc \
PeerInitiateConnectionCommand.h PeerInteractionCommand.cc \
PeerInteractionCommand.h Peer.cc Peer.h \
TorrentDownloadEngine.cc TorrentDownloadEngine.h \
@ -259,31 +259,30 @@ am__libaria2c_a_SOURCES_DIST = Socket.h SocketCore.cc SocketCore.h \
DiskAdaptor.cc DiskAdaptor.h AbstractSingleDiskAdaptor.cc \
AbstractSingleDiskAdaptor.h CopyDiskAdaptor.cc \
CopyDiskAdaptor.h DirectDiskAdaptor.cc DirectDiskAdaptor.h \
MultiDiskAdaptor.cc MultiDiskAdaptor.h FileEntry.cc \
FileEntry.h TrackerUpdateCommand.cc TrackerUpdateCommand.h \
ByteArrayDiskWriter.cc ByteArrayDiskWriter.h \
PeerChokeCommand.cc PeerChokeCommand.h HaveEraseCommand.cc \
HaveEraseCommand.h TorrentRequestInfo.cc TorrentRequestInfo.h \
SeedCriteria.h TimeSeedCriteria.h ShareRatioSeedCriteria.h \
UnionSeedCriteria.h SeedCheckCommand.cc SeedCheckCommand.h \
PeerListProcessor.h DefaultPeerListProcessor.cc \
DefaultPeerListProcessor.h CompactPeerListProcessor.cc \
CompactPeerListProcessor.h DelegatingPeerListProcessor.cc \
DelegatingPeerListProcessor.h AnnounceTier.h AnnounceList.h \
AnnounceList.cc BtContext.h DefaultBtContext.cc \
DefaultBtContext.h PieceStorage.h DefaultPieceStorage.cc \
DefaultPieceStorage.h DefaultPeerStorage.cc \
DefaultPeerStorage.h BtAnnounce.h DefaultBtAnnounce.cc \
DefaultBtAnnounce.h BtRegistry.cc BtRegistry.h BtRuntime.h \
BtProgressInfoFile.h DefaultBtProgressInfoFile.cc \
DefaultBtProgressInfoFile.h BtContextAwareCommand.cc \
BtContextAwareCommand.h BtMessage.h AbstractBtMessage.h \
SimpleBtMessage.cc SimpleBtMessage.h BtAllowedFastMessage.cc \
BtAllowedFastMessage.h BtBitfieldMessage.cc \
BtBitfieldMessage.h BtCancelMessage.cc BtCancelMessage.h \
BtChokeMessage.cc BtChokeMessage.h BtHaveAllMessage.cc \
BtHaveAllMessage.h BtHaveMessage.cc BtHaveMessage.h \
BtHaveNoneMessage.cc BtHaveNoneMessage.h \
MultiDiskAdaptor.cc MultiDiskAdaptor.h TrackerUpdateCommand.cc \
TrackerUpdateCommand.h ByteArrayDiskWriter.cc \
ByteArrayDiskWriter.h PeerChokeCommand.cc PeerChokeCommand.h \
HaveEraseCommand.cc HaveEraseCommand.h TorrentRequestInfo.cc \
TorrentRequestInfo.h SeedCriteria.h TimeSeedCriteria.h \
ShareRatioSeedCriteria.h UnionSeedCriteria.h \
SeedCheckCommand.cc SeedCheckCommand.h PeerListProcessor.h \
DefaultPeerListProcessor.cc DefaultPeerListProcessor.h \
CompactPeerListProcessor.cc CompactPeerListProcessor.h \
DelegatingPeerListProcessor.cc DelegatingPeerListProcessor.h \
AnnounceTier.h AnnounceList.h AnnounceList.cc BtContext.h \
DefaultBtContext.cc DefaultBtContext.h PieceStorage.h \
DefaultPieceStorage.cc DefaultPieceStorage.h \
DefaultPeerStorage.cc DefaultPeerStorage.h BtAnnounce.h \
DefaultBtAnnounce.cc DefaultBtAnnounce.h BtRegistry.cc \
BtRegistry.h BtRuntime.h BtProgressInfoFile.h \
DefaultBtProgressInfoFile.cc DefaultBtProgressInfoFile.h \
BtContextAwareCommand.cc BtContextAwareCommand.h BtMessage.h \
AbstractBtMessage.h SimpleBtMessage.cc SimpleBtMessage.h \
BtAllowedFastMessage.cc BtAllowedFastMessage.h \
BtBitfieldMessage.cc BtBitfieldMessage.h BtCancelMessage.cc \
BtCancelMessage.h BtChokeMessage.cc BtChokeMessage.h \
BtHaveAllMessage.cc BtHaveAllMessage.h BtHaveMessage.cc \
BtHaveMessage.h BtHaveNoneMessage.cc BtHaveNoneMessage.h \
BtInterestedMessage.cc BtInterestedMessage.h \
BtKeepAliveMessage.cc BtKeepAliveMessage.h \
BtNotInterestedMessage.cc BtNotInterestedMessage.h \
@ -336,7 +335,6 @@ am__libaria2c_a_SOURCES_DIST = Socket.h SocketCore.cc SocketCore.h \
@ENABLE_BITTORRENT_TRUE@ CopyDiskAdaptor.$(OBJEXT) \
@ENABLE_BITTORRENT_TRUE@ DirectDiskAdaptor.$(OBJEXT) \
@ENABLE_BITTORRENT_TRUE@ MultiDiskAdaptor.$(OBJEXT) \
@ENABLE_BITTORRENT_TRUE@ FileEntry.$(OBJEXT) \
@ENABLE_BITTORRENT_TRUE@ TrackerUpdateCommand.$(OBJEXT) \
@ENABLE_BITTORRENT_TRUE@ ByteArrayDiskWriter.$(OBJEXT) \
@ENABLE_BITTORRENT_TRUE@ PeerChokeCommand.$(OBJEXT) \
@ -429,6 +427,7 @@ am__objects_3 = SocketCore.$(OBJEXT) Command.$(OBJEXT) \
ChecksumCommand.$(OBJEXT) CheckIntegrityEntry.$(OBJEXT) \
RequestGroupEntry.$(OBJEXT) Cookie.$(OBJEXT) \
CookieParser.$(OBJEXT) CookieBoxFactory.$(OBJEXT) \
HttpHeaderProcessor.$(OBJEXT) FileEntry.$(OBJEXT) \
$(am__objects_1) $(am__objects_2)
am_libaria2c_a_OBJECTS = $(am__objects_3)
libaria2c_a_OBJECTS = $(am_libaria2c_a_OBJECTS)
@ -663,7 +662,9 @@ SRCS = Socket.h SocketCore.cc SocketCore.h Command.cc Command.h \
CheckIntegrityEntry.h CheckIntegrityMan.h ProgressAwareEntry.h \
RequestGroupEntry.cc RequestGroupEntry.h Cookie.cc Cookie.h \
CookieParser.cc CookieParser.h CookieBoxFactory.cc \
CookieBoxFactory.h $(am__append_1) $(am__append_2)
CookieBoxFactory.h HttpHeaderProcessor.cc \
HttpHeaderProcessor.h FileEntry.cc FileEntry.h $(am__append_1) \
$(am__append_2)
noinst_LIBRARIES = libaria2c.a
libaria2c_a_SOURCES = $(SRCS)
aria2c_LDADD = libaria2c.a @LIBINTL@ @ALLOCA@ @LIBGNUTLS_LIBS@\
@ -839,6 +840,7 @@ distclean-compile:
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/HttpConnection.Po@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/HttpDownloadCommand.Po@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/HttpHeader.Po@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/HttpHeaderProcessor.Po@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/HttpInitiateConnectionCommand.Po@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/HttpProxyRequestCommand.Po@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/HttpProxyResponseCommand.Po@am__quote@

17
src/MetalinkEntry.cc
View File

@ -36,11 +36,9 @@
#include "Util.h"
#include <algorithm>
MetalinkEntry::MetalinkEntry():
size(0)
MetalinkEntry::MetalinkEntry()
#ifdef ENABLE_MESSAGE_DIGEST
,
checksum(0),
:checksum(0),
chunkChecksum(0)
#endif // ENABLE_MESSAGE_DIGEST
{}
@ -102,3 +100,14 @@ void MetalinkEntry::dropUnsupportedResource() {
partition(resources.begin(), resources.end(), Supported());
resources.erase(split, resources.end());
}
FileEntries MetalinkEntry::toFileEntry(const MetalinkEntries& metalinkEntries)
{
FileEntries entries;
for(MetalinkEntries::const_iterator itr = metalinkEntries.begin();
itr != metalinkEntries.end(); ++itr) {
entries.push_back((*itr)->file);
}
return entries;
}

28
src/MetalinkEntry.h
View File

@ -39,15 +39,20 @@
#include "MetalinkResource.h"
#include "Checksum.h"
#include "MetalinkChunkChecksum.h"
#include "FileEntry.h"
#include <deque>
class MetalinkEntry;
typedef SharedHandle<MetalinkEntry> MetalinkEntryHandle;
typedef deque<MetalinkEntryHandle> MetalinkEntries;
class MetalinkEntry {
public:
string filename;
FileEntryHandle file;
string version;
string language;
string os;
int64_t size;
ChecksumHandle checksum;
public:
MetalinkResources resources;
@ -56,15 +61,15 @@ public:
#endif // ENABLE_MESSAGE_DIGEST
public:
MetalinkEntry();
~MetalinkEntry();
MetalinkEntry& operator=(const MetalinkEntry& metalinkEntry) {
if(this != &metalinkEntry) {
this->filename = metalinkEntry.filename;
this->file = metalinkEntry.file;
this->version = metalinkEntry.version;
this->language = metalinkEntry.language;
this->os = metalinkEntry.os;
this->size = metalinkEntry.size;
this->checksum = metalinkEntry.checksum;
#ifdef ENABLE_MESSAGE_DIGEST
this->chunkChecksum = metalinkEntry.chunkChecksum;
@ -73,14 +78,23 @@ public:
return *this;
}
string getPath() const
{
return file->getPath();
}
int64_t getLength() const
{
return file->getLength();
}
void dropUnsupportedResource();
void reorderResourcesByPreference();
void setLocationPreference(const string& location, int preferenceToAdd);
static FileEntries toFileEntry(const MetalinkEntries& metalinkEntries);
};
typedef SharedHandle<MetalinkEntry> MetalinkEntryHandle;
typedef deque<MetalinkEntryHandle> MetalinkEntries;
#endif // _D_METALINK_ENTRY_H_

33
src/MetalinkRequestInfo.cc
View File

@ -37,6 +37,7 @@
#include "prefs.h"
#include "DlAbortEx.h"
#include "MultiUrlRequestInfo.h"
#include "Util.h"
class AccumulateNonP2PUrl {
private:
@ -87,19 +88,42 @@ RequestInfos MetalinkRequestInfo::execute() {
printf("No file matched with your preference.\n");
throw new DlAbortEx("No file matched with your preference.");
}
if(op->get(PREF_SHOW_FILES) == V_TRUE) {
Util::toStream(cout, MetalinkEntry::toFileEntry(entries));
return RequestInfos();
}
bool useIndex;
Integers selectIndexes;
Util::unfoldRange(op->get(PREF_SELECT_FILE), selectIndexes);
if(selectIndexes.size()) {
useIndex = true;
} else {
useIndex = false;
}
RequestGroups groups;
int32_t count = 0;
for(MetalinkEntries::iterator itr = entries.begin(); itr != entries.end();
itr++) {
itr++, ++count) {
MetalinkEntryHandle& entry = *itr;
if(op->defined(PREF_METALINK_LOCATION)) {
entry->setLocationPreference(op->get(PREF_METALINK_LOCATION), 100);
}
if(useIndex) {
if(find(selectIndexes.begin(), selectIndexes.end(), count+1) == selectIndexes.end()) {
continue;
}
} else if(!targetFiles.empty()) {
if(find(targetFiles.begin(), targetFiles.end(), entry->getPath()) == targetFiles.end()) {
continue;
}
}
entry->dropUnsupportedResource();
if(entry->resources.size() == 0) {
continue;
}
logger->info("Metalink: Queueing %s for download.",
entry->filename.c_str());
entry->getPath().c_str());
MetalinkResources::iterator itr =
find_if(entry->resources.begin(),
entry->resources.end(),
@ -119,8 +143,9 @@ RequestInfos MetalinkRequestInfo::execute() {
urls.push_back((*itr)->url);
}
RequestGroupHandle rg = new RequestGroup(urls, op);
rg->setHintFilename(entry->filename);
rg->setHintTotalLength(entry->size);
rg->setHintFilename(entry->file->getBasename());
rg->setTopDir(entry->file->getDirname());
rg->setHintTotalLength(entry->getLength());
rg->setNumConcurrentCommand(op->getAsInt(PREF_METALINK_SERVERS));
#ifdef ENABLE_MESSAGE_DIGEST

5
src/MetalinkRequestInfo.h
View File

@ -40,6 +40,7 @@
class MetalinkRequestInfo : public RequestInfo {
private:
string metalinkFile;
Strings targetFiles;
public:
MetalinkRequestInfo(const string& metalinkFile, Option* op):
RequestInfo(op),
@ -47,6 +48,10 @@ public:
virtual ~MetalinkRequestInfo() {}
virtual RequestInfos execute();
void setTargetFiles(const Strings& targetFiles) {
this->targetFiles = targetFiles;
}
};
#endif // _D_METALINK_REQUEST_INFO_H_

2
src/MultiUrlRequestInfo.cc
View File

@ -40,6 +40,8 @@
#include "DNSCache.h"
#include "TorrentRequestInfo.h"
#include "MetalinkRequestInfo.h"
#include "Util.h"
#include <signal.h>
extern volatile sig_atomic_t haltRequested;

1
src/MultiUrlRequestInfo.h
View File

@ -36,6 +36,7 @@
#define _D_MULTI_URL_REQUEST_INFO_H_
#include "RequestInfo.h"
#include "RequestGroup.h"
class MultiUrlRequestInfo : public RequestInfo {
private:

3
src/RequestGroup.cc
View File

@ -53,6 +53,9 @@ SegmentManHandle RequestGroup::initSegmentMan()
{
_segmentMan = _segmentManFactory->createNewInstance();
_segmentMan->ufilename = _ufilename;
if(!_topDir.empty() && _topDir != ".") {
_segmentMan->dir += "/"+_topDir;
}
return _segmentMan;
}

6
src/RequestGroup.h
View File

@ -54,6 +54,7 @@ private:
int64_t _hintTotalLength;
string _hintFilename;
string _ufilename;
string _topDir;
Strings _uris;
Strings _spentUris;
SegmentManHandle _segmentMan;
@ -288,6 +289,11 @@ public:
{
return _gid;
}
void setTopDir(const string& topDir)
{
_topDir = topDir;
}
};
typedef SharedHandle<RequestGroup> RequestGroupHandle;

60
src/RequestInfo.h
View File

@ -38,52 +38,6 @@
#include "common.h"
#include "LogFactory.h"
#include "Option.h"
#include "DownloadEngine.h"
#include "Util.h"
#include "Checksum.h"
#include <signal.h>
class FileInfo {
public:
string filename;
long long int length;
Checksum checksum;
public:
FileInfo():length(0) {}
~FileInfo() {}
bool isEmpty() const {
return filename.size() == 0 && length == 0;
}
bool checkReady() const {
#ifdef ENABLE_MESSAGE_DIGEST
return !isEmpty() && !checksum.isEmpty();
#else
return false;
#endif // ENABLE_MESSAGE_DIGEST
}
bool check() const {
#ifdef ENABLE_MESSAGE_DIGEST
unsigned char md[MAX_MD_LENGTH];
Util::fileChecksum(filename, md, checksum.getDigestAlgo());
return Util::toHex(md,
MessageDigestContext::digestLength(checksum.getDigestAlgo()))
== checksum.getMessageDigest();
#else
return false;
#endif // ENABLE_MESSAGE_DIGEST
}
bool isTorrentFile() const {
return Util::endsWith(filename, ".torrent");
}
bool isMetalinkFile() const {
return Util::endsWith(filename, ".metalink");
}
};
class RequestInfo;
@ -94,8 +48,6 @@ class RequestInfo {
protected:
Option* op;
const Logger* logger;
Checksum checksum;
FileInfo fileInfo;
bool fail;
void printDownloadCompeleteMessage(const string& filename) {
@ -114,21 +66,15 @@ protected:
public:
RequestInfo(Option* op):
op(op),
logger(LogFactory::getInstance()),
fail(false)
{
logger = LogFactory::getInstance();
}
{}
virtual ~RequestInfo() {}
virtual RequestInfos execute() = 0;
bool isFail() const { return fail; }
void setChecksum(const Checksum& checksum) {
this->checksum = checksum;
}
const Checksum& getChecksum() const { return checksum; }
const FileInfo& getFileInfo() const { return fileInfo; }
};
#endif // _D_REQUEST_INFO_H_

4
src/TorrentConsoleDownloadEngine.cc
View File

@ -48,7 +48,7 @@ void TorrentConsoleDownloadEngine::sendStatistics() {
if(pieceStorage->downloadFinished()) {
printf("Download Completed.");
} else {
printf("%s/%sB %d%% %s D:%.2f",
printf("%sB/%sB %d%% %s D:%.2f",
Util::abbrevSize(downloadLength).c_str(),
Util::abbrevSize(totalLength).c_str(),
(totalLength == 0 ?
@ -56,7 +56,7 @@ void TorrentConsoleDownloadEngine::sendStatistics() {
avgSpeed == 0 ? "-" : Util::secfmt(eta).c_str(),
downloadSpeed/1024.0);
}
printf(" U:%.2f(%s) %d peers",
printf(" U:%.2f(%sB) %d peers",
uploadSpeed/1024.0,
Util::abbrevSize(uploadLength).c_str(),
btRuntime->getConnections());

19
src/TorrentRequestInfo.cc
View File

@ -42,6 +42,7 @@
#include "message.h"
#include "RecoverableException.h"
#include "DNSCache.h"
#include <signal.h>
extern volatile sig_atomic_t btHaltRequested;
@ -59,7 +60,7 @@ RequestInfos TorrentRequestInfo::execute() {
btContext->load(torrentFile);
if(op->get(PREF_SHOW_FILES) == V_TRUE) {
showFileEntry(btContext);
Util::toStream(cout, btContext->getFileEntries());
return RequestInfos();
}
// set max_tries to 1. AnnounceList handles retries.
@ -120,19 +121,3 @@ RequestInfos TorrentRequestInfo::execute() {
return RequestInfos();
}
void TorrentRequestInfo::showFileEntry(const BtContextHandle& btContext)
{
FileEntries fileEntries = btContext->getFileEntries();
cout << _("Files:") << endl;
cout << "idx|path/length" << endl;
cout << "===+===========================================================================" << endl;
int count = 1;
for(FileEntries::const_iterator itr = fileEntries.begin();
itr != fileEntries.end(); count++, itr++) {
printf("%3d|%s\n |%s Bytes\n", count,
(*itr)->getPath().c_str(),
Util::llitos((*itr)->getLength(), true).c_str());
cout << "---+---------------------------------------------------------------------------" << endl;
}
}

1
src/TorrentRequestInfo.h
View File

@ -44,7 +44,6 @@ private:
string torrentFile;
Strings targetFiles;
void showFileEntry(const BtContextHandle& btContext);
public:
TorrentRequestInfo(const string& torrentFile, Option* op):
RequestInfo(op),

19
src/Util.cc
View File

@ -47,6 +47,7 @@
#include <arpa/inet.h>
#include <unistd.h>
#include <signal.h>
#include <iomanip>
template<typename T>
@ -705,8 +706,8 @@ string Util::abbrevSize(int64_t size)
for(; i < numUnit-1 && size >= 1024; ++i) {
r = size&0x3ff;
size >>= 10;
}
return Util::llitos(size, true)+"."+Util::itos(r*10/1024)+units[i];
}
return Util::llitos(size, true)+"."+Util::itos(r*10/1024)+units[i]+"i";
}
time_t Util::httpGMT(const string& httpStdTime)
@ -717,3 +718,17 @@ time_t Util::httpGMT(const string& httpStdTime)
time_t thetime = timegm(&tm);
return thetime;
}
void Util::toStream(ostream& os, const FileEntries& fileEntries)
{
os << _("Files:") << "\n";
os << "idx|path/length" << "\n";
os << "===+===========================================================================" << "\n";
int count = 1;
for(FileEntries::const_iterator itr = fileEntries.begin();
itr != fileEntries.end(); count++, itr++) {
os << setw(3) << count << "|" << (*itr)->getPath() << "\n";
os << " |" << Util::llitos((*itr)->getLength(), true) << " bytes" << "\n";
os << "---+---------------------------------------------------------------------------" << "\n";
}
}

6
src/Util.h
View File

@ -36,6 +36,7 @@
#define _D_UTIL_H_
#include "common.h"
#include "FileEntry.h"
#ifdef ENABLE_MESSAGE_DIGEST
#include "messageDigest.h"
#endif // ENABLE_MESSAGE_DIGEST
@ -44,8 +45,7 @@
#include <deque>
#include <sys/time.h>
#include <stdio.h>
using namespace std;
#include <ostream>
#define STRTOLL(X) strtoll(X, (char**)NULL, 10);
@ -152,6 +152,8 @@ public:
static string abbrevSize(int64_t size);
static time_t httpGMT(const string& httpTimeFormat);
static void toStream(ostream& os, const FileEntries& entries);
};
#endif // _D_UTIL_H_

12
src/Xml2MetalinkProcessor.cc
View File

@ -98,13 +98,15 @@ MetalinkEntryHandle Xml2MetalinkProcessor::getEntry(const string& xpath) {
MetalinkEntryHandle entry(new MetalinkEntry());
entry->filename = filename;
FileEntryHandle fileEntry = new FileEntry(filename, 0, 0);
string sizeStr = Util::trim(xpathContent(xpath+"/m:size"));
if(sizeStr == "") {
entry->size = 0;
fileEntry->setLength(0);
} else {
entry->size = STRTOLL(sizeStr.c_str());
fileEntry->setLength(strtoll(sizeStr.c_str(), 0, 10));
}
entry->file = fileEntry;
entry->version = Util::trim(xpathContent(xpath+"/m:version"));
entry->language = Util::trim(xpathContent(xpath+"/m:language"));
entry->os = Util::trim(xpathContent(xpath+"/m:os"));
@ -127,9 +129,9 @@ MetalinkEntryHandle Xml2MetalinkProcessor::getEntry(const string& xpath) {
string sha1PiecesPath = piecesPath+"[@type=\"sha1\"]";
string md5PiecesPath = piecesPath+"[@type=\"md5\"]";
if(xpathExists(sha1PiecesPath)) {
entry->chunkChecksum = getPieceHash(sha1PiecesPath, entry->size);
entry->chunkChecksum = getPieceHash(sha1PiecesPath, entry->getLength());
} else if(xpathExists(md5PiecesPath)) {
entry->chunkChecksum = getPieceHash(md5PiecesPath, entry->size);
entry->chunkChecksum = getPieceHash(md5PiecesPath, entry->getLength());
}
#endif // ENABLE_MESSAGE_DIGEST
for(int index = 1; 1; index++) {

34
src/main.cc
View File

@ -37,7 +37,6 @@
#include "Util.h"
#include "prefs.h"
#include "FeatureConfig.h"
#include "UrlRequestInfo.h"
#include "MultiUrlRequestInfo.h"
#include "TorrentRequestInfo.h"
#include "BitfieldManFactory.h"
@ -211,16 +210,28 @@ void showUsage() {
" multiple URIs for a single entity: deliminate\n"
" URIs by Tab in a single line.") << endl;
cout << _(" -j, --max-concurrent-downloads=N Set maximum number of concurrent downloads.\n"
" It should be used with -i option.\n"
" Default: 5") << endl;
cout << _(" --load-cookies=FILE Load cookies from FILE. The format of FILE is\n"
" one used by Netscape and Mozilla.") << endl;
#if defined ENABLE_BITTORRENT || ENABLE_METALINK
cout << _(" -S, --show-files Print file listing of .torrent or .metalink file\n"
" and exit.") << endl;
cout << _(" --select-file=INDEX... Set file to download by specifing its index.\n"
" You can know file index through --show-files\n"
" option. Multiple indexes can be specified by using\n"
" ',' like \"3,6\".\n"
" You can also use '-' to specify rangelike \"1-5\".\n"
" ',' and '-' can be used together.\n"
" When used with -M option, index may vary depending\n"
" on the query(see --metalink-* options).") << endl;
#endif // ENABLE_BITTORRENT || ENABLE_METALINK
#ifdef ENABLE_BITTORRENT
cout << _(" -T, --torrent-file=TORRENT_FILE The file path to .torrent file.") << endl;
cout << _(" --follow-torrent=true|false Setting this option to false prevents aria2 to\n"
" enter BitTorrent mode even if the filename of\n"
" downloaded file ends with .torrent.\n"
" Default: true") << endl;
cout << _(" -S, --show-files Print file listing of .torrent file and exit.") << endl;
cout << _(" --direct-file-mapping=true|false Directly read from and write to each file\n"
" mentioned in .torrent file.\n"
" Default: true") << endl;
@ -230,12 +241,6 @@ void showUsage() {
" 0 means unrestricted.\n"
" You can append K or M(1K = 1024, 1M = 1024K).\n"
" Default: 0") << endl;
cout << _(" --select-file=INDEX... Set file to download by specifing its index.\n"
" You can know file index through --show-files\n"
" option. Multiple indexes can be specified by using\n"
" ',' like \"3,6\".\n"
" You can also use '-' to specify rangelike \"1-5\".\n"
" ',' and '-' can be used together.") << endl;
cout << _(" --seed-time=MINUTES Specify seeding time in minutes. See also\n"
" --seed-ratio option.") << endl;
cout << _(" --seed-ratio=RATIO Specify share ratio. Seed completed torrents until\n"
@ -405,16 +410,18 @@ int main(int argc, char* argv[]) {
{ "input-file", required_argument, 0, 'i' },
{ "max-concurrent-downloads", required_argument, 0, 'j' },
{ "load-cookies", required_argument, &lopt, 205 },
#if defined ENABLE_BITTORRENT || ENABLE_METALINK
{ "show-files", no_argument, NULL, 'S' },
{ "select-file", required_argument, &lopt, 21 },
#endif // ENABLE_BITTORRENT || ENABLE_METALINK
#ifdef ENABLE_BITTORRENT
{ "torrent-file", required_argument, NULL, 'T' },
{ "listen-port", required_argument, &lopt, 15 },
{ "follow-torrent", required_argument, &lopt, 16 },
{ "show-files", no_argument, NULL, 'S' },
{ "no-preallocation", no_argument, &lopt, 18 },
{ "direct-file-mapping", required_argument, &lopt, 19 },
// TODO remove upload-limit.
//{ "upload-limit", required_argument, &lopt, 20 },
{ "select-file", required_argument, &lopt, 21 },
{ "seed-time", required_argument, &lopt, 22 },
{ "seed-ratio", required_argument, &lopt, 23 },
{ "max-upload-limit", required_argument, &lopt, 24 },
@ -709,7 +716,7 @@ int main(int argc, char* argv[]) {
Util::setGlobalSignalHandler(SIGPIPE, SIG_IGN, 0);
RequestInfo* firstReqInfo = 0;
RequestInfo* firstReqInfo;
#ifdef ENABLE_BITTORRENT
if(op->defined(PREF_TORRENT_FILE)) {
firstReqInfo = new TorrentRequestInfo(op->get(PREF_TORRENT_FILE),
@ -726,6 +733,11 @@ int main(int argc, char* argv[]) {
if(op->defined(PREF_METALINK_FILE)) {
firstReqInfo = new MetalinkRequestInfo(op->get(PREF_METALINK_FILE),
op);
Strings targetFiles;
if(op->defined(PREF_METALINK_FILE) && !args.empty()) {
targetFiles = args;
}
((MetalinkRequestInfo*)firstReqInfo)->setTargetFiles(targetFiles);
}
else
#endif // ENABLE_METALINK