Proposal: Twitter archival bot (Developer needed!)
-
This is a proposal for a Twitter/X bot which aims to assist in archiving relevant Twitter content by backing it up to the forum.
By tagging the bot account at the end of a thread, the preceding posts in the thread would get copied and reposted to the forum, just like e.g. the existing Thread Reader App bot.
The motivation for having such a bot is twofold:
- Content on Twitter is fragile. Accounts frequently get deleted or banned and all posts by that account get deleted in the process.
- It's a way to kickstart the forum and increase it's search engine ranking.
There are a few challenges that need to be addressed:
- Avoiding duplicates: If the bot is tagged multiple times in the same post in a thread, or if it is tagged multiple times in different posts in a thread, a new thread on the forum should probably not be created.
- Spam: Since anyone can tag the bot account in any post, I suggest that the bot has a way of filtering out irrelevant content. This can be done by: some combination of keywords checks, account whitelists and LLM algorithms.
If you feel up for the task of developing the bot, this forum provides a very nice API:
I suggest making it open-source so that other developers can help improve upon the bot.
Discuss!
-
@pilky This is an awesome proposal, I think it's a great idea. There are also search issues with Twitter that a forum seems better equipped to handle.
-
@pilky thank you alima
-
P pilky pinned this topic on Jan 30, 2024, 1:44 PM
-
Great idea @pilky. I'll take a look at these API docs.
The knowing could come in handy for other ideas. Imagine. A private peatbot, fine tuned on everything, including retrieval from complete copies of the same documents used to fine tune it for proper citation. You could call it in to a thread to have it comment in context.
-
Maybe some will find this a useful starting point.
After brief but traumatic experience with Twitter API I decided to use ChatGPT to simulate it instead so the code might be not fully correct or optimal as I wasn't able to test it properly. Also, there are many things that need to be added e.g. how to handle quote tweets, images, create tags and titles, or handle errors; that I felt inadequate to address.
-
@zawisza This is great, I can def. see this being incorporated.
Read access through the Twitter API costs 100$/month, they removed free access last year.
I guess the bot would have to do some scraping to get the mentions and thread contents, I'm not sure if Twitter limits that somehow.
The bot could still tweet a reply linking to the reposted forum thread. -
I have zero knowledge of bot building BUT I have been able to create some Google Aps scripts with help from my new best friend AI CHAT GPT. I also had zero knowledge of Aps scripts before hand.
Chat GPT could be useful.