[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Wp-mirror-list] Attempting to Mirror en
From: |
Benjamin Goldsmith |
Subject: |
[Wp-mirror-list] Attempting to Mirror en |
Date: |
Tue, 25 Sep 2012 16:44:42 -0400 |
Hello Dr. Miller:
I'm back. Now I'm trying to mirror English Wikipedia without images using
wp-mirror.
Since downloading images is time consuming, I'd like to have a mirror without
images that, potentially, could link to the existing images at Wikipedia.
Though
this would not be a full mirror, it would provide text. In the meantime I
could have
another computer download the complete mirror.
I will also try to remove the links to the other Wikipedia versions on my
mirror.
I have hit a roadblock. Here are where things stand:
1) Running Debian GNU Linux 6 Squeeze:
* Dell 2950 PowerEdge Server
* 500 GB RAM (no images will be downloaded on this server)
* 8 GB RAM
* 2 CPU threads
* Wired internet, no proxy
2) Installed wp-mirror_0.3-2_all.deb and all dependencies following the
instructions. However, I added the following to /etc/wp-mirror/local.conf
(defparameter *mirror-image-download-p* nil)
(defparameter *mirror-image-validate-p* nil)
3) Performed extra steps as I did during my work with the simple mirror on
my laptop to get wp-mirror to run:
a) created the wpmirror database by hand and granted permissions
on it to 'wikiuser'@'localhost' (
b) commented out lines 3443-3456 of wpmirror
c) removed the & from line 1196
I also had to edit line 1335 to up the timeout in shell-curl-http-file
from 5000. The download of the big pages-articles gz was timing out
with about 10 minutes left to go.
4) Ran wp-mirror. It ran to completion.
5) Now when I point my browser to http://en.mediawiki.site/ I get a blank
screen.
6) MySQL
It looks like the database was created, the xchunks generated but the pages were
not loaded:
address@hidden:~# mysql --host=localhost --user=wikiuser -p
<snip>
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| enwiki |
| wikidb |
| wpmirror |
+--------------------+
4 rows in set (0.00 sec)
mysql> SELECT language,type,state,COUNT(*) FROM wpmirror.file GROUP BY
language,type,state;
+----------+----------+-------+----------+
| language | type | state | COUNT(*) |
+----------+----------+-------+----------+
| en | checksum | done | 1 |
| en | dump | done | 1 |
| en | xml | done | 1 |
| en | xchunk | valid | 12681 |
+----------+----------+-------+----------+
4 rows in set (0.01 sec)
mysql> SELECT COUNT(*) FROM enwiki.page;
+----------+
| COUNT(*) |
+----------+
| 1 |
+----------+
1 row in set (0.00 sec)
7) mediawiki
As expected, the xchunks were generated:
address@hidden:/var/lib/mediawiki/images/wp-mirror# ls
enwiki-20120902-pages-articles-p000000000-c000001000.xml
enwiki-20120902-pages-articles-p000001000-c000001000.xml
<snip about 12.6k lines>
enwiki-20120902-pages-articles-p012679000-c000001000.xml
enwiki-20120902-pages-articles-p012680000-c000001000.xml
enwiki-20120902-pages-articles.xml
enwiki-20120902-pages-articles.xml.bz2
enwiki-20120902-pages-articles.xml.bz2.head
enwiki-latest-md5sums.txt
enwiki-latest-md5sums.txt.head
address@hidden:~# ls -la /usr/share/mediawiki/maintenance/importDump_farm.php
-rw-r--r-- 1 root root 182 Mar 6 2012
/usr/share/mediawiki/maintenance/importDump_farm.php
8) wp-mirror
In /var/log/wp-mirror.log, none of the steps [fail]. They are all [pass]
or [done].
What are my next steps?
Thank you again for your time and attention.
Best,
Ben
<<winmail.dat>>
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- [Wp-mirror-list] Attempting to Mirror en,
Benjamin Goldsmith <=