This repo holds a simple script that mirrors what payed zurg does to ingest into zilean. It is meant to be used with aiostreams or another indexer for stremio/jellyfin.
In here are also many of my own notes that are (probably) not documented in other places. This should be very useful for anyone running their own zilean instance.
This works REALLY well for real debrid. If using all debrid, I would reccomend using mediafusion from midnightignight or elfhosted (and select only cached torrents).
If you have read this before, please read the A Simpler Solution section.
Why do this? Public scrapers are typically inconsistent and overloaded. If you're sick of that, this is an extremely reliable and fast alternative. Now only you can mess up your streaming service! This way, you can host a stremio or jellyfin/plex setup where you self host everything except for debrid. Furthermore, this scrapes Debrid Media Manager (DMM), and so far I have not come accross any broken links. This is coming from someone who used to run many public scrapers through the aiostreams service.
Now, DMM has more cached options than torrentio, elfhosted, and others, but still some extremely niche searches will still not be available. That's where zurg fits in. Zurg is used to hold an index of just your DMM library. Zilean will then scrape from zurg an index it with the rest of the results. This means you can use stremio/jellyfin, and if anything ever isn't available, simply go to DMM and add that torrent to your list. Then zilean will update with that torrent, and you can stream it.
When running zilean, use this fork by Ayusha. And star it. It will scrape DMM over github, the way it was intended to be safely scraped, and will not ingest 10gb a day. Instead, it will just pull the changes. I built most of these notes by reading through the commit history on that project. It is much much faster and will not get you rate limited like normal zilean will.
Use the following yml. It's mostly based off of these docs, however edited to use the fork and my own fake zurk. Use this yml, and add your github token to it for max speed. You don't need to give your token any permissions. Also, uncomment the lines in the environment if you wish to use zurg too.
git clone https://github.com/nasawyer7/fakezurg.git
Use the examplezilean.yml file. Run it with docker, and use this as your zilean setup. Within about 36 hours, depending on your internet, you should have around 1,250,000 torrents.
If you do not wish to add zurg into this mix, simply stop here (or maybe after the next section). Do not uncomment the below lines. Connect aiostreams or your favorite indexer to your new zilean instance (seriously, aiostreams works great, just use that). Then add aiostreams to stremio or use the Gelato plugin to add it to plex (superior option imo).
Zilean has some strange behavior. While zilean is supposed to be edited with the settings.json file, however I never got this file to function correctly. When binded to the host, it will constantly reset. If overwritten inside the container with cat, it will reset as well. I only was able to edit these settings useing environment variables, which is exactly why I included my docker config. That file is here:
docker exec -it zilean cat /app/data/settings.json
Anyways, useful documentation is here. You don't need to stop zilean to start the scraper with this command:
docker exec -d zilean /app/scraper dmm-sync
Use -t if you wish to watch it. To see the number of torrents you have cached (so far I have 1,287,350):
docker exec -it zilean-postgres psql -U postgres -d zilean -c "SELECT count(*) FROM \"Torrents\";"
And to sync with your local Zurg instance:
docker exec -t zilean /app/scraper generic-sync
To view logs without the noisy scraping logs:
docker logs zilean | grep -E -v 'Filtered out|Processing file|Removed duplicate hashes'
If you wish to simply configure jellyfin/plex to view your local media directly, then use these commands from the official Zurg free repo:
Clone the repo git clone https://github.com/debridmediamanager/zurg-testing.git or git clone https://github.com/debridmediamanager/zurg.git
Add your token in config.yml
sudo mkdir -p /mnt/zurg
Run docker compose up -d
time ls -1R /mnt/zurg You're done! If you do edits on your config.yml just do docker compose restart zurg.
They forgot one command lol. chmod 666 /mnt/zurg. Then just allow your jellyfin/plex setup to see the mnt zurg as a media location.
If you wish to use Gelato plugin on jellyfin, or use stremio, use my setup. I beleive paid Zurg would also allow you to do this? Not entirely sure, I never tested it. FreeZurg serves files in html, which zilean cannot parse correctly. I have a python script that will simply return all of the cached torrents on your debrid account. It's simple. You can just run it using tmux or screen, set it up as a systemd service, or compile as a docker file. I compiled it as a docker file because I love docker for running big services, but its up to you. simply git clone this repo, then run:
sudo docker compose up -d --build
And that's all! Check to see if any files were served and indexed with docker logs fakezurg.
I lied, you need to add your real debrid API key to this :) (you can also set this up with all debrid if you want, but WAY more ppl that use DMM are on real debrid).
Make sure you update environment variables in Zilean to point to where fakezurg is running. Currently, zilean pulls from fakezurg every 5 minutes, but you can change that.
If you want anything niche that is not available, simply add the torrent to DMM. With this config, all of your services (stremio, jellyfin, plex) will include your niche media now.
After more testing with Zurg, I found that it does not actually update changes made to hashlists with this fork I referenced. It only pulls new hashlists. To force changes from hashlists, run this command. You can add this to a cron job to run every 30 minutes or hour so your Zilean stays constantly updated with the DMM lists.
docker exec -it zilean-postgres psql -U postgres -d zilean -c 'TRUNCATE TABLE "ParsedPages";'
For stremio, aiostreams is extremly easy to setup. I would recomend selfhosting aiostreams, as it's built in proxy and ability to add other sources as well are amazing. Simply add a zilean instance, and choose your new subdomain for zilean once it's port forwarded and created.
With Gelato, there is a little more work that needs to be done. TMDB must be added to receive metadata, and the version selection tells you almost nothing about the stream. Here is my custom config that shows what I think is the most important information on each jellyfin stream:
{addon.name} - {stream.size::bytes} - {stream.resolution} - {stream.audioChannels} - {stream.visualTags} - {stream.languages} -{service.cached::istrue["Cached"||""]}{service.cached::isfalse["unstable {stream.seeders}"||""]}
Place this in the AIOstreams formatter section. I would also reccomend enableing all hardware transcoding settings that you can, especially color mapping to fix hdr color wash out on non hdr devices.
