How To Run USDownloader On Ubuntu Server 8.10

To run the best in automati? file share downloading - russian Universal Share Downloader (USD)  (http://www.dimonius.ru/) on a GUI-less (or, they say, headless) Linux server (in my case it's Ubuntu Server 8.10) we will need four steps. The most complicated thing will be setting up the "fake" framebuffer version of X - Xvfb. The rest is easy.

Well, let's go:

 

1) - SET UP XVFB, WINE and CABEXTRACT -

== Install Xvfb, wine and cabextract:

$ sudo apt-get install xvfb wine cabextract

Due to a minor packaging bug in xvfb (https://bugs.launchpad.net/ubuntu/+source/xorg-server/+bug/294454) we need to compensate it by installing some additional packages:

$ sudo apt-get install xfs xfonts-scalable xfonts-100dpi

== Start Xvfb:

$ /usr/bin/X11/Xvfb :10.10 -screen 10 800x600x16 -ac -br -kb -c -fbdir /var/tmp/ &

** At this step you may get several errors with default Xorg configuration (due to mentioned bug in xvfb package) - so here are for reference my conf files:

$ cat /etc/X11/xorg.conf

  # xorg.conf (X.Org X Window System server configuration file)
   #
   # This file was generated by dexconf, the Debian X Configuration tool, using
   # values from the debconf database.
   #
   # Edit this file with caution, and see the xorg.conf manual page.
   # (Type "man xorg.conf" at the shell prompt.)
   #
   # This file is automatically updated on xserver-xorg package upgrades *only*
   # if it has not been modified since the last upgrade of the xserver-xorg
   # package.
   #
   # If you have edited this file but would like it to be automatically updated
   # again, run the following command:
   # sudo dpkg-reconfigure -phigh xserver-xorg

Section "InputDevice"
   Identifier "Generic Keyboard"
   Driver "kbd"
   Option "XkbRules" "xorg"
   Option "XkbModel" "pc105"
   Option "XkbLayout" "us,ru"
   Option "XkbVariant" ",winkeys"
   Option "XkbOptions" "grp:caps_toggle,grp_led:scroll"
   EndSection

Section "InputDevice"
   Identifier "Configured Mouse"
   Driver "mouse"
   Option "CorePointer"
   EndSection

Section "Device"
   Identifier "Intel Corporation 82845G/GL[Brookdale-G]/GE Chipset Int egrated Graphics Device"
   EndSection

Section "Monitor"
   Identifier "LG"
   EndSection

Section "Screen"
   Identifier "Default Screen"
   Device "Intel Corporation 82845G/GL[Brookdale-G]/GE Chipset Integrated Graph ics Device"
   Monitor "LG"
   DefaultDepth 16
   SubSection "Display"
   Viewport 0 0
   Depth 16
   Modes "800x600" "640x480"
   EndSubSection
   EndSection

Section "ServerLayout"
   Identifier "Default Layout"
   Screen "Default Screen"
   EndSection
 

$ cat /etc/X11/xserver/SecurityPolicy

  version-1

# $Xorg: SecurityPolicy,v 1.3 2000/08/17 19:47:56 cpqbld Exp $

# The site policy fields are interpreted by the XC-QUERY-SECURITY-1
   # authorization protocol. The values are arbitrary and site-specific.
   # Refer to the Security Extension Specification for the usage of the policies.
   #sitepolicy A
   #sitepolicy B
   #sitepolicy C

# Property access rules:
   # property <property> <window> <permissions>
   # <window> ::= any | root | <propertyselector>
   # <propertyselector> ::= <property> | <property>=<value>
   # <permissions> :== [ <operation> | <action> | <space> ]*
   # <operation> :== r | w | d
   # r read
   # w write
   # d delete
   # <action> :== a | i | e
   # a allow
   # i ignore
   # e error

# Allow reading of application resources, but not writing.
   property RESOURCE_MANAGER root ar iw
   property SCREEN_RESOURCES root ar iw

# Ignore attempts to use cut buffers. Giving errors causes apps to crash,
   # and allowing access may give away too much information.
   property CUT_BUFFER0 root irw
   property CUT_BUFFER1 root irw
   property CUT_BUFFER2 root irw
   property CUT_BUFFER3 root irw
   property CUT_BUFFER4 root irw
   property CUT_BUFFER5 root irw
   property CUT_BUFFER6 root irw
   property CUT_BUFFER7 root irw

# If you are using Motif, you probably want these.
   property _MOTIF_DEFAULT_BINDINGS root ar iw
   property _MOTIF_DRAG_WINDOW root ar iw
   property _MOTIF_DRAG_TARGETS any ar iw
   property _MOTIF_DRAG_ATOMS any ar iw
   property _MOTIF_DRAG_ATOM_PAIRS any ar iw

# If you are running CDE you also need these
   property _MOTIF_WM_INFO root arw
   property TT_SESSION root irw
   property WM_ICON_SIZE root irw
   property "SDT Pixel Set" any irw

# The next two rules let xwininfo -tree work when untrusted.
   property WM_NAME any ar

# Allow read of WM_CLASS, but only for windows with WM_NAME.
   # This might be more restrictive than necessary, but demonstrates
   # the <required property> facility, and is also an attempt to
   # say "top level windows only."
   property WM_CLASS WM_NAME ar

# These next three let xlsclients work untrusted. Think carefully
   # before including these; giving away the client machine name and command
   # may be exposing too much.
   property WM_STATE WM_NAME ar
   property WM_CLIENT_MACHINE WM_NAME ar
   property WM_COMMAND WM_NAME ar

# To let untrusted clients use the standard colormaps created by
   # xstdcmap, include these lines.
   property RGB_DEFAULT_MAP root ar
   property RGB_BEST_MAP root ar
   property RGB_RED_MAP root ar
   property RGB_GREEN_MAP root ar
   property RGB_BLUE_MAP root ar
   property RGB_GRAY_MAP root ar

# To let untrusted clients use the color management database created
   # by xcmsdb, include these lines.
   property XDCCC_LINEAR_RGB_CORRECTION root ar
   property XDCCC_LINEAR_RGB_MATRICES root ar
   property XDCCC_GRAY_SCREENWHITEPOINT root ar
   property XDCCC_GRAY_CORRECTION root ar

# To let untrusted clients use the overlay visuals that many vendors
   # support, include this line.
   property SERVER_OVERLAY_VISUALS root ar
 

== Set up the current display:

$ export DISPLAY=:10.10

Doublecheck it:

$ echo $DISPLAY

== Dance around to get Xvfb work correct (at least - wine-applications shouldn't  crash with critical error).

Minor errors (fonts-related and like) won't shut down USDownloader.

Try to start, for example:

$ wine notepad.exe &

And check if it's running:

$ ps -C notepad.exe 

 

2) - SET UP IES4LINUX -

Follow the instructions from the respectable author (I guess step ?6 is the only we really need for Ubuntu 8.10): http://www.tatanka.com.br/ies4linux/page/Installation:Ubuntu

 

3) - SET UP WINDOWS SCRIPTING HOST (needed for .VBS) -

== Download this great script for automatic installation of windoze components on wine (big thanks to input of Dan Kegel, [email protected]):

http://kegel.com/wine/winetricks

We need to hack this script a bit - removing with  /Q option all unnecessary (in our headless :) case) pop-ups with user action request ("Accept license terms" etc):

Change corresponding line from>

  try $WINE "$WINETRICKS_CACHE"/vc6redistsetup_enu.exe "/T:`$WINE winepath -w "$WINETRICKS_TMP"`" /c $WINETRICKS_QUIET

to>

  try $WINE "$WINETRICKS_CACHE"/vc6redistsetup_enu.exe "/T:`$WINE winepath -w "$WINETRICKS_TMP"`" /c /Q

Change corresponding line from>

  $WINE "$WINETRICKS_CACHE"/vcredist.exe || true

to>

  $WINE "$WINETRICKS_CACHE"/vcredist.exe /Q || true 

change corresponding line from>

  try $WINE "$WINETRICKS_CACHE"/WindowsXP-Windows2000-Script56-KB917344-x86-enu.exe $WINETRICKS_QUIET

to>

  try $WINE "$WINETRICKS_CACHE"/WindowsXP-Windows2000-Script56-KB917344-x86-enu.exe /Q

== Install Windows Scripting Host (it's necessary to run .vbs scripts in USD):

$ ./winetricks -v wsh56

 

4) - SET UP USDownloader itself -

== Download the recent portable blackmanos' USD assembly  (we need a JavaScript enabled browser here - assembly is placed on several file-sharing hostings):

http://forum.ru-board.com/topic.cgi?forum=5&topic=27428&start=20#19

Unpack assembly in windoze, copy it's content to some Linux directory under wine (for me it's ~/.wine/drive_c/usd).

== Create  "windoze disk" u:\ as the soft link to the directory where we want USD to put downloaded files then (for my case - it's /mm/usd):

$ cd ~/.wine/dosdevices
$  ln -s -T /mm/usd u:

== Correct corresponding paths in USDownloader.ini (and at this step let's change USD's default language to English - to read logs more easily on Linux):

  [Main]
   SelectLang=1
   Language=USDownloader.eng.lng
   SaveTo=u:\
   SaveToItems=u:\
   ...
 DefaultSaveToFolder=u:\

== Also in USDownloader.ini set up the  USD webserver start:

  [WebServer]
   Enabled=1
   Address=0.0.0.0
   Port=8088
   Login=YOURLOGIN
   Password=YOURPASSWORD
   Form.AddLink.Width=400
   Form.AddLink.URL.Height=200
 Form.AddLink.Descr.Height=100

YOURLOGIN and YOURPASSWORD, of course, should be yours. Don't forget to open the mentioned port (8088 here) in your firewall for outbound connections - it will be the only way to manage USD.

== Start USDownloader:

wine C:\\usd\\USDownloader.exe &

Check if this process is running:

$ ps -C USDownloader.exe

== Use your favorite browser to log in to USD:

http://your.server.ip.address:8088

Check how the stuff works - add some download, simultaneously watching USD log:

$ tail -f ~/.wine/drive_c/usd/USDownloader.log

Downloads from deposit files work fine - therefore it seems that vbs-automation scripts work as they should. That's it.

Share this page:

5 Comment(s)

Add comment

Comments

From: Anonymous

Hi,

 and thanks for pointing me to JDownloader.org. I would never have know of it's existence were it not for the comments-field of this article. Great stuff :)

From: Fred Thompson

"the best"?!?!...those are big words. Compare to jDownloader http://jdownloader.org/home which is platform-independent Java and supports 90 file sharing hosts.

From: Anonymous

Well, thanks for the howto. I got some points which stop me from even trying that tool: - The site is Russian only, I unfortunately can't even read a single word - Howtoforge is a site for Linux howtos and tutorials, so why should I install IE, Windows Scripting Host, Wine etc. to run a windows only tool? I'd rather take Torrentflux or Torrentflux-b4rt which runs natively on Linux - "To run the best in automati? file share downloading" is not very objective

From: Anonymous

Why not use native application like JDownloader

http://jdownloader.org/

From: Christopher Atanasopulo

Really big words for the title, I have to admit. But it is my conclusion from my experience using this kind of software.

 I have a PentiumII box running ubuntu server 9.04, with just 256Mb of RAM. A really old (exactly14 years old) piece of hardware you can say. But she's been racked on a shelf at my room, serving stuff for years (after several system upgrades and reinstallations). I only power it down to open the case once in several monthes to celan some dust off it. It has only a keyboard and ethernet connection. I access it over ssh (I can't remember the last time I attached a monitor to it. I don't know if the graphics card even works).

If I try to use jdownloader, it is useless. Jdownloader does not offer a CLI, nor web interface. Only way to use it is through a vnc connection (purposeless waste of resources). And moreover, it runs over java, more purposeless waste of resources (ok, you can argue that comparing java to wine, but it is not the same in the end).

 If I try to use other cli-centered and very new software, for the purpose it works. But these are very reduced in functionalities. Will get the stuff downloaded for you, yes. Even break captchas, if you want (hope it isn't illegal in your country). But again, no web interface, nor easy handling of batch downloads. You end up writing your own scripts, or altering the software code (most of them are perl or bash scripts) to get what you need, and I find it very unpractical to my current needs (though I've learn a lot this way).

USDownloader was the difference. I can install it without graphic stuff. It will run over wine very very smoothly, even on old hardware. Xvfb is not cpu-hungry. And I can configure the software directly editing just ONE file or even from within its native web interface. Something unthinkable to achieve with jdownloader because it stores configuration in a binary format inside java classes files. I can add, remove, pause, or start, downloads with a click from everywhere just using a web browser over the Internet. It is exactly the thing I was needing.

 I know, it is closed source, its documentation and support forums are in russian, and most people do not trust russians' software (even a russian friend of mine). You can even say that running windows software over wine, having native linux (java is not native though) code out there, is a waste. Well if it works for me, on the described box, I'd find really hard to believe it won't work on ANY current desktop computer, no matter how low-end it might be.

Well, to finish, this is not a troll post. I don't have the intention to start flame wars. I just had the need to put these thoughts toghether and expalin the result of my own experience.

 Thank you ais77 for the grat work on this guide! :)

 PS: Some time ago I used your guide, without asking, as the baseline for writing a guide (adding a couple of things) on my blog in spanish (sorry for that, I'm planning an english version soon).

 PS2: Some packaged distributions (my comment is long enough yet as to explain this) of USDownloader need mono to run some .net code. I added on my guide the way to do it quietly, since it is not contemplated among winetrick options. The change would be the following:

 Change:

 try $WINE "$WINETRICKS_CACHE"/mono-2.2-gtksharp-2.12.7-win32-5.exe

 For:

 try $WINE "$WINETRICKS_CACHE"/mono-2.2-gtksharp-2.12.7-win32-5.exe /SP- /VERYSILENT /SUPPRESSMSGBOXES

 Was hard to find, but it is covered here: https://bugzilla.novell.com/show_bug.cgi?id=450347