Quoted By: >>7905
The Bibliotheca Anonoma has built two drop in replacements in Python to replace the aging FoolFuuka/Asagi archival stack. https://wiki.bibanon.org/FoolFuuka
Our goal is to increase scraper efficiency, increase performance of the search engine, and significantly reduce the training time and setup necessary to operate a 4chan archive.
This new server setup allows us to generate backups without affecting scraping, public backups will be uploaded over the coming months.
If you'd like to help develop or plan to make a 4chan archive of your own, it is a tough road but we have the expertise to help you set things up. Post below or contact us at:
irc.rizon.net #bibanon
https://matrix.to/#/#bibanon-chat:matrix.org
https://discord.com/invite/3jxxGDC
* https://github.com/bibanon/neofuuka-scraper is in use at Desuarchive as part of this server migration. See the improved features over Asagi scraper in the repository.
* https://github.com/bibanon/ayase is a drop in replacement for the FoolFuuka frontend, check it out at our demonstration site: https://ayase.wakarimasen.moe. It is sufficient for non-ghostposting use, but we are still working on search and moderation tools and development help is appreciated.
Lastly, We hope to learn Rust to totally redesign the Asagi SQL schema. (currently dependent on Percona MariaDB TokuDB and has numerous conversion issues) Two efforts are underway:
* https://github.com/miyachan/torako is a drop in replacement for the Asagi scraper. It is in testing at https://ayase.wakarimasen.moe .
* https://github.com/reasv/mitsuba is a full stack archive stack replacement and PostgreSQL schema redesign, most likely the best path forward past the aging Asagi schema.
Our goal is to increase scraper efficiency, increase performance of the search engine, and significantly reduce the training time and setup necessary to operate a 4chan archive.
This new server setup allows us to generate backups without affecting scraping, public backups will be uploaded over the coming months.
If you'd like to help develop or plan to make a 4chan archive of your own, it is a tough road but we have the expertise to help you set things up. Post below or contact us at:
irc.rizon.net #bibanon
https://matrix.to/#/#bibanon-chat:matrix.org
https://discord.com/invite/3jxxGDC
* https://github.com/bibanon/neofuuka-scraper is in use at Desuarchive as part of this server migration. See the improved features over Asagi scraper in the repository.
* https://github.com/bibanon/ayase is a drop in replacement for the FoolFuuka frontend, check it out at our demonstration site: https://ayase.wakarimasen.moe. It is sufficient for non-ghostposting use, but we are still working on search and moderation tools and development help is appreciated.
Lastly, We hope to learn Rust to totally redesign the Asagi SQL schema. (currently dependent on Percona MariaDB TokuDB and has numerous conversion issues) Two efforts are underway:
* https://github.com/miyachan/torako is a drop in replacement for the Asagi scraper. It is in testing at https://ayase.wakarimasen.moe .
* https://github.com/reasv/mitsuba is a full stack archive stack replacement and PostgreSQL schema redesign, most likely the best path forward past the aging Asagi schema.
