6 Commits

Author SHA1 Message Date
5dc996a1b7 add github action to run the script 2024-01-16 16:40:16 +05:30
8e70a7f467 fix logic for filtering posts 2024-01-16 16:37:30 +05:30
c26e85aa07 add empty list check 2024-01-14 20:29:30 +05:30
77af1679c8 change title to a better version 2024-01-14 14:57:03 +05:30
0161160431 rename func 2024-01-14 14:54:34 +05:30
360e1aa292 add community roundup script 2024-01-14 14:44:07 +05:30
17 changed files with 117 additions and 386 deletions

View File

@ -1,11 +1,9 @@
name : Community Threads Wiki Updater
on:
schedule:
- cron: '0 0 * * *' # This cron expression triggers the workflow every day at midnight UTC
workflow_dispatch:
inputs:
post_url:
description: 'The URL of the Reddit post to add'
required: true
permissions:
contents: read
@ -34,4 +32,4 @@ jobs:
GIST_TOKEN: ${{ secrets.GIST_TOKEN }}
run: |
cd community-threads
python main.py ${{ github.event.inputs.post_url }}
python main.py

View File

@ -2,6 +2,7 @@
> Bunch of scripts to automate stuff in r/developersIndia.
[![Discord](https://img.shields.io/discord/669880381649977354?color=%237289da&label=Discord&logo=Discord)](https://discordapp.com/invite/MKXMSNC)
[![Subreddit subscribers](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fdevelopersindia.github.io%2Fmetrics%2Fdata%2F&query=%24.totalMembers&suffix=%20members&style=flat&logo=reddit&label=r%2FdevelopersIndia&color=orange&link=https%3A%2F%2Fwww.reddit.com%2Fr%2FdevelopersIndia
)](https://www.reddit.com/r/developersIndia/)
@ -24,20 +25,15 @@
- Used for grabbing the posts from [community threads collection](https://www.reddit.com/r/developersIndia/collection/958aef35-f9cb-414d-ab33-08bc639e47de/) and adding it to the [wiki](https://www.reddit.com/r/developersIndia/wiki/community-threads/).
- ![action build](https://github.com/developersIndia/deviras/actions/workflows/collection-thread-updater.yml/badge.svg)
### [job-thread](https://github.com/developersIndia/deviras/blob/main/job-thread/main.py)
### [job_thread](https://github.com/developersIndia/deviras/blob/main/job_thread/main.py)
- Used for creating [hiring threads](https://www.reddit.com/r/developersIndia/?f=flair_name%3A%22Hiring%22) in the subreddit that gets the job from our [job board](https://developersindia.in/job-board/).
Used for creating [hiring threads](https://www.reddit.com/r/developersIndia/?f=flair_name%3A%22Hiring%22) in the subreddit that gets the job from our [job board](https://developersindia.in/job-board/).
### [showcase-sunday](https://github.com/developersIndia/deviras/blob/main/showcase-sunday/main.py)
- Used for creating [Showcase Sunday Megathreads](https://www.reddit.com/r/developersIndia/search/?q=flair%3A%20Showcase%20Sunday&restrict_sr=1) posts in the subreddit.
- ![action build](https://github.com/developersIndia/deviras/actions/workflows/showcase-sunday.yml/badge.svg)
### [ama-summarizer](https://github.com/developersIndia/deviras/blob/main/ama-summarizer/main.py/)
- The Python script to help during AMAs. It generates a markdown file of questions and links of the questions the AMA guest has answered.
## Setup
1. Clone the repo
@ -79,10 +75,8 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
<td align="center" valign="top" width="14.28%"><a href="https://animesh-ghosh.github.io/"><img src="https://avatars.githubusercontent.com/u/34956994?v=4?s=100" width="100px;" alt="MaDDogx"/><br /><sub><b>MaDDogx</b></sub></a><br /><a href="https://github.com/developersIndia/deviras/commits?author=Animesh-Ghosh" title="Code">💻</a> <a href="https://github.com/developersIndia/deviras/commits?author=Animesh-Ghosh" title="Tests">⚠️</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://pratham.cc"><img src="https://avatars.githubusercontent.com/u/67585967?v=4?s=100" width="100px;" alt="Pratham"/><br /><sub><b>Pratham</b></sub></a><br /><a href="https://github.com/developersIndia/deviras/commits?author=git-bruh" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://nisarga.me/"><img src="https://avatars.githubusercontent.com/u/45588772?v=4?s=100" width="100px;" alt="Nisarga Adhikary"/><br /><sub><b>Nisarga Adhikary</b></sub></a><br /><a href="https://github.com/developersIndia/deviras/commits?author=ni5arga" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://lineararray.nekoweb.org"><img src="https://github.com/lineararray.png" width="100px;" alt="LinearArray"/><br /><sub><b>LinearArray</b></sub></a><br /><a href="https://github.com/developersIndia/deviras/commits?author=LinearArray" title="Code">💻</a></td>
</tr>
</tbody>
</table>
<!-- markdownlint-restore -->

View File

@ -1 +0,0 @@
questions.md

View File

@ -1,65 +0,0 @@
import praw
import os
def get_reddit_instance():
# Reddit API credentials
user_agent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 14.2; rv:109.0) Gecko/20100101 Firefox/121.0"
client_id = os.environ["REDDIT_CLIENT_ID"]
client_secret = os.environ["REDDIT_CLIENT_SECRET"]
reddit_pass = os.environ["REDDIT_PASSWORD"]
username = os.environ["REDDIT_USERNAME"]
# Create a Reddit instance
reddit = praw.Reddit(
client_id=client_id,
client_secret=client_secret,
password=reddit_pass,
user_agent=user_agent,
username=username,
)
return reddit
def get_post_url():
post_url = input("Enter the AMA post URL: ") # reddit.com URLs preferred
return post_url
def get_guest_username():
guest_username = input("Enter the AMA guest username: ")
return guest_username
def main():
reddit = get_reddit_instance()
post_url = get_post_url()
guest_username = get_guest_username()
submission = reddit.submission(url=post_url)
submission.comments.replace_more(limit=None)
markdown_file = ""
question_number = 1
for comment in submission.comments.list():
if comment.author and comment.author.name.lower() == guest_username.lower():
# TODO truncate long questions with ellipsis
question_text = comment.parent().body.replace("\n", " ")
# avoid deleted questions/comments
if question_text != "[deleted]":
question_link = "https://reddit.com" + comment.parent().permalink
markdown_file += (
f"{question_number}. [{question_text}]({question_link})\n\n"
) # Add an extra newline after each question
question_number += 1
with open("questions.md", "w", encoding="utf-8") as file:
file.write(markdown_file)
print(f"{question_number - 1} questions generated successfully.")
if __name__ == "__main__":
main()

View File

@ -1,5 +0,0 @@
# ama-summarizer
- The Python script to help during [AMAs](https://developersindia.in/ama-archive/). It generates a markdown file of questions and links to the questions the AMA guest has answered.
- This script is designed to generate a Markdown file containing questions and links from a Reddit post's comments. It specifically focuses on questions or comments answered by a specific user.
- The resulting Markdown file will contain a list of questions or comments that the specified guest has answered, with each question numbered and linked to the corresponding Reddit comment.

View File

@ -1 +0,0 @@
praw

View File

@ -1,7 +1,6 @@
import datetime
import praw
import os
import sys
import json
import requests
@ -21,73 +20,6 @@ def is_last_day_of_month():
return tomorrow.day == 1
def get_posts_by_flair(subreddit, flair):
current_year = datetime.date.today().year
current_month = datetime.date.today().month
posts = []
for post in subreddit.search(f'flair_name:"{flair}"', time_filter="month"):
post_date = datetime.datetime.fromtimestamp(post.created_utc)
if post_date.year == current_year and post_date.month == current_month:
post.title = post.title.replace("|", "\\|") # Escape the "|" character
posts.append(post)
posts = sorted(posts, key=lambda post: post.created_utc, reverse=True)
return posts
def get_weekly_discussion_posts(subreddit):
flair = next(
filter(
lambda flair: "Weekly Discussion" in flair["flair_text"],
subreddit.flair.link_templates.user_selectable(),
)
)
return get_posts_by_flair(subreddit, flair["flair_text"])
def get_ama_posts(subreddit):
flair = next(
filter(
lambda flair: "AMA" in flair["flair_text"],
subreddit.flair.link_templates.user_selectable(),
)
)
return get_posts_by_flair(subreddit, flair["flair_text"])
def get_i_made_this_posts(subreddit):
flair = next(
filter(
lambda flair: "I Made This" in flair["flair_text"],
subreddit.flair.link_templates.user_selectable(),
)
)
# Get all posts with the specified flair
posts = get_posts_by_flair(subreddit, flair["flair_text"])
# Sort the posts by upvotes and then comments in descending order
posts = sorted(
posts, key=lambda post: (post.score, post.num_comments), reverse=True
)
# Return only the top 10 posts
return posts[:10]
def get_announcement_posts(subreddit):
flair = next(
filter(
lambda flair: "Announcement" in flair["flair_text"],
subreddit.flair.link_templates.user_selectable(),
)
)
return get_posts_by_flair(subreddit, flair["flair_text"])
def get_gist_content(gist_id):
headers = {
"Authorization": f"token {token}",
@ -99,33 +31,24 @@ def get_gist_content(gist_id):
return gist["files"][filename]["content"]
def get_community_threads():
def get_monthly_roundup():
saved_collection_posts = json.loads(get_gist_content(gist_id))
# filter posts for this month & year
saved_collection_posts = list(
filter(
lambda post: datetime.datetime.strptime(
post["created_at"], "%Y-%m-%dT%H:%M:%S"
).year
== datetime.date.today().year
and datetime.datetime.strptime(
).year == datetime.date.today().year and
datetime.datetime.strptime(
post["created_at"], "%Y-%m-%dT%H:%M:%S"
).month
== datetime.date.today().month,
).month == datetime.date.today().month,
saved_collection_posts["posts"],
)
)
return saved_collection_posts
def create_community_roundup_post(
subreddit,
posts,
i_made_this_posts,
weekly_discussion_posts,
ama_posts,
announcement_posts,
):
def create_community_roundup_post(subreddit, posts):
flair = next(
filter(
lambda flair: "Community Roundup" in flair["flair_text"],
@ -137,49 +60,18 @@ def create_community_roundup_post(
month=datetime.date.today().strftime("%B"), year=datetime.date.today().year
)
text = "|||"
text += "\n|--------|--------|\n"
footer_text = """\n\n
---
**Community Roundup is posted on the last day of each month. To explore a compilation of all interesting posts and community threads over time, [visit our wiki](https://www.reddit.com/r/developersIndia/wiki/community-threads/).**\n
The collection is curated by our volunteer team & is independent of the number of upvotes and comments (except for "I made This" posts). If you believe we may have overlooked any engaging posts or discussions, please share them with us via [modmail](https://reddit.com/message/compose?to=r/developersIndia&subject=Community%20Threads%20Collection%20Suggestion&message=Hey%20folks%2C%0A%0A%3Cpost%20link%3E).\n
**Community Roundup is posted on the last day of every month. You can find the [schedule on our events calendar](https://developersindia.in/events-calendar). To find the list of all [interesting posts & community threads over time, checkout our wiki](https://www.reddit.com/r/developersIndia/wiki/community-threads/).\n**
If you think we missed any interesting post or discussion, please share it with us via modmail.
"""
if len(announcement_posts) > 0:
text = "## Announcements\n||\n|--------|\n"
for post in announcement_posts:
text += f"| [**{post.title}**]({post.url}) |\n"
else:
print("No announcements found. Skipping")
if len(ama_posts) > 0:
text += "\n## AMAs\n||\n|--------|\n"
for post in ama_posts:
text += f"| [**{post.title}**]({post.url}) |\n"
else:
print("No AMAs found. Skipping")
if len(posts) > 0:
text += "\n## Community Threads\n|S.No|Discussions started by members|\n|--------|--------|\n"
posts_counter = 0
for post in posts:
posts_counter += 1
text += f"| {posts_counter} | [**{post['title']}**]({post['url']}) |\n"
else:
print("No posts found in the collection for this month. Skipping")
if len(weekly_discussion_posts) > 0:
text += "\n## Weekly Discussions\n|Started by Volunteer/Mod Team|\n|--------|\n"
for post in weekly_discussion_posts:
text += f"| [**{post.title}**]({post.url}) |\n"
else:
print("No weekly discussions found. Skipping")
if len(i_made_this_posts) > 0:
text += "\n## I Made This\n|Top 10 posts|\n|--------|\n"
for post in i_made_this_posts:
text += f"| [**{post.title}**]({post.url}) |\n"
else:
print("No I Made This posts found. Skipping")
text = text + footer_text
@ -208,14 +100,11 @@ def main():
subreddit = reddit.subreddit(sub)
if is_last_day_of_month():
posts = get_community_threads()
i_made_this_posts = get_i_made_this_posts(subreddit)
weekly_discussion_posts = get_weekly_discussion_posts(subreddit)
ama_posts = get_ama_posts(subreddit)
announcement_posts = get_announcement_posts(subreddit)
create_community_roundup_post(
subreddit, posts, i_made_this_posts, weekly_discussion_posts, ama_posts, announcement_posts
)
posts = get_monthly_roundup()
if len(posts) == 0:
print("No posts found in the collection for this month. Skipping")
return
create_community_roundup_post(subreddit, posts)
print("Community Roundup post created successfully!")
else:
print("Skipping. Not the last day of the month")

View File

@ -1,6 +1,5 @@
import praw
import os
import argparse
from datetime import datetime
import json
from collections import defaultdict
@ -37,26 +36,13 @@ def update_gist(gist_id, filename, content, description=""):
)
return response.json()
# farewell, reddit collections
# def get_collection(reddit):
# collection = reddit.subreddit(sub).collections(
# permalink="https://reddit.com/r/developersIndia/collection/958aef35-f9cb-414d-ab33-08bc639e47de"
# )
# return collection
def get_post_data(reddit, post_url):
submission = reddit.submission(url=post_url)
post = {
"title": submission.title,
"url": submission.url,
"id": submission.id,
"num_comments": submission.num_comments,
"created_at": datetime.utcfromtimestamp(
submission.created_utc
).isoformat(),
"flair_text": submission.link_flair_text,
}
return post
def get_collection(reddit):
collection = reddit.subreddit(sub).collections(
permalink="https://reddit.com/r/developersIndia/collection/958aef35-f9cb-414d-ab33-08bc639e47de"
)
return collection
def update_wiki(reddit, wikipage, posts):
# Group posts by year
@ -75,7 +61,7 @@ def update_wiki(reddit, wikipage, posts):
wiki_header = """# A collection of must read discussions started by community members"""
content = wiki_header + "\n\n"
content += f"A handpicked collection of **{total_posts}** interesting posts, discussions & high-quality threads gathered over **{total_years-1}** years & counting.\n\n"
content += f"A handpicked collection of **{total_posts}** interesting posts, discussions & high-quality threads gathered over **{total_years}** years & counting.\n\n"
content += "If you spot a post that could be in this list, send us a [modmail](https://reddit.com/message/compose?to=r/developersIndia&subject=Community%20Threads%20Collection%20Suggestion&message=Hey%20folks%2C%0A%0A%3Cpost%20link%3E)\n\n"
for year in sorted(posts_by_year.keys(), reverse=True):
@ -92,10 +78,6 @@ def update_wiki(reddit, wikipage, posts):
def main():
parser = argparse.ArgumentParser(description='Update Community Threads Collection.')
parser.add_argument('post_url', help='The URL of the Reddit post to add.')
args = parser.parse_args()
reddit = praw.Reddit(
client_id=client_id,
client_secret=client_secret,
@ -104,13 +86,38 @@ def main():
user_agent=f"Automod reader by u/{username}",
)
collection = get_collection(reddit)
saved_collection_posts = json.loads(get_gist_content(gist_id))
saved_collection_ids = [post["id"] for post in saved_collection_posts["posts"]]
print(f"Database was last updated on {saved_collection_posts['collection_last_updated']}")
print(f"Collection was last updated on {datetime.utcfromtimestamp(collection.last_update_utc).isoformat()}")
if (
saved_collection_posts["collection_last_updated"]
!= datetime.utcfromtimestamp(collection.last_update_utc).isoformat()
):
print("Collection was updated, getting new posts data...")
# given 2 lists find non-common elements
db_posts = set(saved_collection_ids)
collection_posts = []
for submission in collection:
collection_posts.append(submission.id)
collection_posts = set(collection_posts)
new_posts = list(collection_posts - db_posts)
deleted_posts = list(db_posts - collection_posts)
print(f"Found {len(new_posts)} new posts!")
print(f"Found {len(deleted_posts)} deleted posts!")
posts = []
# load the saved collection posts data
for submission_id in saved_collection_posts["posts"]:
if submission_id["id"] in deleted_posts:
continue
post = {
"title": submission_id["title"],
"url": submission_id["url"],
@ -121,13 +128,28 @@ def main():
}
posts.append(post)
new_post = get_post_data(reddit, args.post_url)
if new_post["id"] not in saved_collection_ids:
posts.append(new_post)
# get the new posts data
for submission_id in new_posts:
submission = reddit.submission(submission_id)
post = {
"title": submission.title,
"url": submission.url,
"id": submission.id,
"num_comments": submission.num_comments,
"created_at": datetime.utcfromtimestamp(
submission.created_utc
).isoformat(),
"flair_text": submission.link_flair_text,
}
posts.append(post)
# sort the posts by created_at
posts = sorted(posts, key=lambda k: k["created_at"])
collection_json = {
"collection_last_updated": datetime.utcnow().isoformat(),
"collection_last_updated": datetime.utcfromtimestamp(
collection.last_update_utc
).isoformat(),
"posts": posts,
}
@ -135,7 +157,7 @@ def main():
print("Internal database updated successfully!")
update_wiki(reddit, "community-threads", posts)
else:
print("Post is already in the collection. No changes were made.")
print("Wiki is up to date!")
if __name__ == "__main__":

View File

@ -1,2 +0,0 @@
praw
feedparser

View File

@ -180,9 +180,6 @@ def create_job_post(subreddit) -> Post:
flair_id=flair["flair_template_id"],
)
submission.mod.sticky()
submission.mod.distinguish()
submission.mod.approve()
submission.mod.lock()
return Post(post_id=submission.id, epoch=submission.created_utc)

View File

@ -1,8 +1,7 @@
import os
import sys
import praw
import time
import datetime
client_id = os.environ["REDDIT_CLIENT_ID"]
client_secret = os.environ["REDDIT_CLIENT_SECRET"]
@ -11,61 +10,12 @@ username = os.environ["REDDIT_USERNAME"]
sub = "developersIndia"
def get_last_activity_times(reddit, username):
user = reddit.redditor(username)
# Get the user's last comment time in the subreddit
last_comment_time = None
for comment in user.comments.new(
limit=100
): # look at the user's 100 most recent comments
if comment.subreddit.display_name == sub:
last_comment_time = datetime.datetime.fromtimestamp(comment.created_utc).strftime('%d %B, %Y')
break
# Get the user's last post creation time and title in the subreddit
last_post_time = None
last_post_title = None
for submission in user.submissions.new(
limit=100
): # look at the user's 100 most recent posts
if submission.subreddit.display_name == sub:
last_post_time = datetime.datetime.fromtimestamp(submission.created_utc).strftime('%d %B, %Y')
last_post_title = submission.title
break
return last_comment_time, last_post_time, last_post_title
def get_current_flair(reddit, username):
subreddit = reddit.subreddit(sub)
flair = next(subreddit.flair(username))
template = get_flair_template_from_text(reddit, flair["flair_text"])
if template is None:
return None, None
return flair["flair_text"], template["id"]
def get_flair(reddit, username):
subreddit = reddit.subreddit(sub)
flair = next(subreddit.flair(username))
template = get_template_from_flair_text(reddit, flair["flair_text"])
if template is None:
return None, None
return flair["flair_text"], template["id"]
def assign_user_flair(reddit, username, flair_text):
subreddit = reddit.subreddit(sub)
flair = next(subreddit.flair(username))
template = get_flair_template_from_text(reddit, flair["flair_text"])
# append YoE to the flair text
verified_text = f"{flair['flair_text']} | {flair_text}"
subreddit.flair.set(username, text=verified_text, flair_template_id=template["id"])
template = get_flair_template_from_text(reddit, flair['flair_text'])
subreddit.flair.set(username, text=flair_text, flair_template_id=template['id'])
def get_flair_templates(reddit):
subreddit = reddit.subreddit(sub)
@ -75,15 +25,7 @@ def get_flair_templates(reddit):
def get_flair_template_from_text(reddit, flair_text):
templates = get_flair_templates(reddit)
for template in templates:
if template["text"] == flair_text:
return template
def get_template_from_flair_text(reddit, flair_text):
templates = get_flair_templates(reddit)
for template in templates:
# check if the flair text is in the template
if template["text"] in flair_text:
if template['text'] == flair_text:
return template
@ -105,9 +47,7 @@ PS: This was an automated messaage, no need to reply. [Reach out to mods](https:
Namaste 🙏
"""
reddit.redditor(username).message(
subject=message_subject, message=message_text.format(flair=flair_text), from_subreddit=reddit.subreddit(sub)
)
reddit.redditor(username).message(subject=message_subject, message=message_text.format(flair=flair_text))
def main():
@ -128,42 +68,8 @@ def main():
# get flair text from CLI args
flair_text = sys.argv[2]
# get last activity times
print(f"Getting last activity times for {reddit_username}...")
last_comment_time, last_post_time, last_post_title = get_last_activity_times(reddit, reddit_username)
if last_comment_time is not None:
print(f"{reddit_username}'s last comment time on developersIndia was {last_comment_time}")
if last_post_time is not None:
print(f"{reddit_username}'s last post on developersIndia was \"{last_post_title}\" on {last_post_time}")
# get current flair
current_flair_text, current_flair_template_id = get_flair(reddit, reddit_username)
# TODO figure out final flair text
if "Verified" in current_flair_text or "YoE" in current_flair_text:
print(f"{reddit_username} is already verified")
sys.exit(0)
if current_flair_text is None and current_flair_template_id is None:
print(f"{reddit_username} does not have a flair on r/developersIndia")
sys.exit(0)
else:
print(f"{reddit_username}'s current flair is \"{current_flair_text}\", template id: {current_flair_template_id}")
# ask for user input
user_input = input(f"Do you want to verify {reddit_username}? [Y/n]: ")
if user_input.lower() != 'y':
print("Cancelled verification operation.")
sys.exit(0)
assign_user_flair(reddit, reddit_username, flair_text)
# Ya I know, just don't ask
time.sleep(2)
updated_flair_text, _ = get_flair(reddit, reddit_username)
print(f"Updated {reddit_username}'s flair to \"{updated_flair_text}\"")
send_message(reddit, reddit_username, updated_flair_text)
print(f"Sent verification confirmation message to {reddit_username}")
# send_message(reddit, reddit_username)
if __name__ == "__main__":

View File

@ -1 +0,0 @@
praw