|
Poster:
|
 tracey pooh |
Date:
|
October 06, 2009 10:02:08am |
|
Forum:
|
forums
|
Subject:
|
Re: Stale .pureftpd-upload* file blocks checkin |
there can be massive contention for the disks with heavy fragmented reading especially at times. so the speed can be be very bad at times.
we have mostly given up on FTP technology here at archive for exactly the reasons you point out -- why is it going so slow when i can add a second uploader and get 2x the throughput? (that *is* interesting 8-) why are the disks not maxxed out fully by the ftp application? why does resumption not work? all these questions and more were unanswerable by myself and our staff so we have made our single remaining FTP "WIGIWYG", unfortunately ("what you get is what you get").
nonideal to be sure -- but our pure-ftpd server program has had so many problems that we've stopped trying to fix and/or patch around it.
another option is S3 (HTTP PUT) that we support...
-tracey
|
Poster:
|
aronsson |
Date:
|
October 06, 2009 10:18:35am |
|
Forum:
|
forums
|
Subject:
|
Re: Stale .pureftpd-upload* file blocks checkin |
My friends mentioned the TCP window scaling option. With my ping roundtrip time of 194 ms (from 130.236.254.105 in Sweden) TCP without window scaling might max out at 320 kbyte/s.
It is on at my end. I'm running a standard Ubuntu Linux 9.04 and "cat /proc/sys/net/ipv4/tcp_window_scaling" answers "1" (meaning true/on).
|
Poster:
|
Shiva Ho |
Date:
|
October 13, 2009 10:14:19am |
|
Forum:
|
forums
|
Subject:
|
Re: Stale .pureftpd-upload* file blocks checkin |
The "NEW" system has even more flaws & times out repeatedly making it unusable...
Lets FACE it FLASH SUCKS ALWAYS HAS & ALWAYS WILL...
If you can't do it with straight html & java then stop wasting our time & Bandwidth
|
Poster:
|
Shiva Ho |
Date:
|
October 13, 2009 10:16:20am |
|
Forum:
|
forums
|
Subject:
|
FLASH SUCKS |
Your rational is flawed!!! FTP WORKS & Flash doesn't!!!!
|
Poster:
|
tracey pooh |
Date:
|
October 13, 2009 11:36:51am |
|
Forum:
|
forums
|
Subject:
|
Re: FLASH SUCKS |
then don't use flash! 8-)
there is a "prefer to not use flash?" link
on the uploader (new item) page and
one on the edit item files (for an existing item) page.
--tracey
|
Poster:
|
kzns.pl |
Date:
|
October 26, 2009 08:56:57am |
|
Forum:
|
forums
|
Subject:
|
Re: Stale .pureftpd-upload* file blocks checkin |
Hi.
Is there a way to automatic upload of a file to add to an item I've already created? (ie use external program to upload)
We have a community radio (internet based) and are give a .wav file recorded by journalists, and local artists.
We have an in-house specialist computer program which with one-click takes the file, converts it to .mp3, adds ID3 tags, adds it to our database, converts it to text, and creates a webpage for it and (up to now) uploads the mp3 to a server (so the non-technical staff do not have to waste their time doing what a computer can do). Also it means technical staff can concentrate on more taxing issues than upload a file.
We would like to use the Internet Archive for this as it is the best or proper place for it for archiving purposes, but... what technical methods exist, however complex, for automating the process of getting that mp3 file from an inhouse computer to a published and accessible version on Internet Archive?
There were some methods floating around (using php) to upload uears ago when the FTP was all working well.
kzns.pl
|
Poster:
|
tracey pooh |
Date:
|
November 02, 2009 06:30:55pm |
|
Forum:
|
forums
|
Subject:
|
Re: Stale .pureftpd-upload* file blocks checkin |
yes there is -- i can try to put you in touch w/ an engineer here who can point you to S3. you'd want something like curl (ideally) or wget on a unix-like cmd-line if that's possible. then it's relatively straightforward from there.
we can proceed if that sounds good/plausible and you reply!
-tracey
|
Poster:
|
tracey pooh |
Date:
|
November 02, 2009 06:33:46pm |
|
Forum:
|
forums
|
Subject:
|
Re: Stale .pureftpd-upload* file blocks checkin |
yes there is -- i can try to put you in touch w/ an engineer here who can point you to S3. you'd want something like curl (ideally) or wget on a unix-like cmd-line if that's possible. then it's relatively straightforward from there.
we can proceed if that sounds good/plausible and you reply!
-tracey
|
Poster:
|
Nemo_bis |
Date:
|
November 15, 2010 12:22:58pm |
|
Forum:
|
forums
|
Subject:
|
Re: Stale .pureftpd-upload* file blocks checkin |
> I'm still puzzled by the low throughput on FTP uploads. I
> never see more than 319 kbyte/s per session, but I can
> double that by running two parallel sessions. Is somebody
> looking at that?
I tried and used Flash but it's even worse:
http://www.archive.org/post/333852/maximum-upload-speed-flashNow I'm uploading with FTP: 9 parallel uploads (which seem to be the maximum, and yet enough to be often flooded with ECONNRESET error) are now at about 1,5 MiB/s on average, with peaks of 2 MiB/s and much more rarely 3 MiB/s, which may be the actual limit on my end (although it should be higher).
This post was modified by Nemo_bis on 2010-11-15 20:22:58
|
Poster:
|
Nemo_bis |
Date:
|
November 17, 2010 06:43:03am |
|
Forum:
|
forums
|
Subject:
|
Re: Stale .pureftpd-upload* file blocks checkin |
Well, now FileZilla has gone crazy for some minutes because after a "time expired for connection" error which aborted the uploads it couldn't recconnect because of ECONNRESET errors and a "421 Too many connections (10) from this IP" error; I've raised the time between failed login retries from 10 to 60 s and it has recovered after three minutes. Now trying with 8 parallel uploads.
TVNews
OpenLibrary