import Apikeys import requests import pprint import csv import pandas as pd pp = pprint.PrettyPrinter(indent=4) APIKEY = Apikeys.ANTHOLOGY HEADERS = { "accept": "application/json", "X-Api-Key": APIKEY, } BASEURL = "https://api.northpass.com/v2/" GROUPS = [ "Student", ] def all_groups(): """ Get all Group IDs """ gids = [] for group in GROUPS: group_count = 0 while True: group_count += 1 url = f"https://api.northpass.com/v2/groups?filter[name][cont]={group}&page={group_count}" gresponse = requests.get(url, headers=HEADERS) gdata = gresponse.json() gnext = gdata["links"] for response in gdata["data"]: group_id = response["id"] group_name = response["attributes"]["name"] group_tupe = (group_id, group_name) print(group_tupe) gids.append(group_tupe) if "next" not in gnext: break get_courses(gids) def group_ids(): """ Get the group Ids from a list of named groups. """ url = "groups" ids = [] for group in GROUPS: filter = f"?filter[name][cont]={group} - (" urlreq = f"{BASEURL}{url}{filter}" response = requests.get(urlreq, headers=HEADERS) response = response.json() for data in response["data"]: group = data["attributes"]["name"] id1 = data["id"] group_tupe = (id1, group) ids.append(group_tupe) get_courses(ids) #portfolio t1 - dc50ca43-5071-45b3-bf42-e1e64416ffd0 #academy use/nav t1 - 3875a210-f86d-4b4a-a86a-980022374936 def get_courses(ids): df = pd.DataFrame() courses = [] error_groups = [] for tupe in ids: id = tupe[0] group = tupe[1] for count in range(1, 10): print(f"Get Courses For Loop. Group:{ group } and count: {count}") url2 = f"groups/{id}/courses?limit=100&page={count}" courseurl = f"{BASEURL}{url2}" coursereq = requests.get(courseurl, headers=HEADERS) try: coursedata = coursereq.json() nextlink = coursedata["links"] for cdata in coursedata["included"]: # if cdata["attributes"]["status"] == "archived": coursedict = parse_data(cdata, group) print(f"Coursedict --> {coursedict} for page {count}") if coursedict is not None: courses.append(coursedict) except JSONDecodeError as e: print("JSON Error! {e}") error_groups.append(group) finally: if "next" not in nextlink: break pddata = df.from_records(courses) pddata.to_csv("/Users/normrasmussen/Downloads/ALL-Anthology_Course_IDs_Names.csv") print(error_groups) def parse_data(cdata, group): name = cdata["attributes"]["name"] courseid = cdata["attributes"]["share_course_link"].split("/")[5] enrollments = cdata["attributes"]["enrollments_count"] created = cdata["attributes"]["created_at"] updated = cdata["attributes"]["updated_at"] status = cdata["attributes"]["status"] coursedict = { "group": group, "name": name, "id": courseid, "enrollments": enrollments, "created at": created, "updated at": updated, "status": status, } if coursedict is not None: return coursedict if __name__ == "__main__": # group_ids() all_groups()