Re: processing files that are put on the server using SFTP

[Ben Sturmfels]

Hi Lloyd,

On Wed, 06 Oct 2021, Lloyd Kvam wrote:

> I moved mediagoblin to a new server and upgraded to the current
> version. The webserver is serving files properly. However I am unable
> to add new media files.
>
> The old server allowed my daughter's family to upload photos to their
> SFTP account. A symbolic link (since lost) tied their SFTP folders into
> the mediagoblin folder tree. There was a cron job that ran 
> mg_addmedia.py every five minutes. So the old work flow simply required
> that they put photos (and movies) into the SFTP folders and then in a
> few minutes, the files would be available through the web interface.

Sounds like a really intuitive set-up you had going there! I've
previously had a similar arrangement using syncthing to accept local
files and synchronise to the server and a cron-job to ingest them on the
server using the `gmg addmedia` command. My setup was a bit fragile, but
worked really nicely.

I can't see any reason your setup shouldn't continue to work - your
setup may need to be updated to use `gmg addmedia`

eg. `gmg addmedia yourusername your_media.jpg`

You might also like to look at `gmg batchaddmedia` which can accept
multiple files a spreadsheet of metadata. Use `--help` with either
command for help.

> Now when I use the web interface and click "Add media", I get 403
> Forbidden. It's a GET /submit/ request so I assume that's Python code
> and not a file. Any suggestions would be appreciated.

Hmm, that's no good. If your setup is as per the deployment guide, run
the following command in a terminal and try to browse to /submit/ again
to see if you get a useful error:

sudo journalctl -u mediagoblin-paster.service -f

(The deployment guide:
https://docs.mediagoblin.org/en/master/siteadmin/deploying.html)

If there's nothing interesting there, the other place I'd look would be
the Nginx log, which will be something like:

sudo tail -f /var/log/nginx/mediagoblin.example.error.log

> Where does mediagoblin look for new files to process? How should that
> kind of automatic processing work today?

MediaGoblin doesn't have a default place to look - you need to provide the path
explicitly to `gmg addmedia`. After that it will copy them into the
`user_dev/media` directory.

Let us know how you go.

Regards,
Ben

Ben Sturmfels

Sturm Software Engineering
www.sturm.com.au
+61 3 9024 2467