Data collection

[saurabh-khanna]

Data we can collect using public resources

1. Engagement Metrics:

   • Data: Likes, comments, and shares for posts/accounts.
   • Source: Instagram Graph API (limited to authenticated accounts with permissions).
   • Python Resources:
     • instagrapi: A Python library for Instagram automation and data retrieval.
     • requests/urllib for direct API calls.

2. Metadata:

   • Data: Post timing, hashtags, geotags, follower counts, bio details.
   • Source: Instagram Graph API or publicly visible profile information (if scraping is allowed).
   • Python Resources:
     • instagrapi or beautifulsoup4 for web scraping (if permitted).

3. Content Data:

   • Data: Post captions, hashtags, or image/video metadata.
   • Source: Instagram Graph API or scraping public posts.
   • Python Resources:
     • instagrapi for API-based caption and hashtag extraction.
     • Pillow or OpenCV for analyzing downloaded images.

4. Hashtag and Topic Trends:

   • Data: Frequency and reach of hashtags or topics.
   • Source: Manually collect hashtags or use third-party tools.

5. Sentiment and Keyword Analysis:

   • Data: Extracted captions and comments for text analysis.
   • Python Resources:
     • NLTK or spaCy for NLP tasks like sentiment analysis or keyword extraction.
     • Transformers from Hugging Face for advanced sentiment or topic modeling.

6. Sensitive Content Analysis:

   • Data: Analyze engagement trends for flagged sensitive content.
   • Source: Measure differences pre- and post-flagging using API data.

7. Historical Policy Data:

   • Data: Changes in Instagram's guidelines over time.
   • Source: Archive platforms like the Wayback Machine or scraping public policy pages.
   • Python Resources:
     • selenium or requests for automating data retrieval from policy archives.

8. Adversarial Testing Data:

   • Data: Results from posting test content systematically (different hashtags, metadata).
   • Source: Automated posting experiments.
   • Python Resources:
     • instagrapi or custom bots for automated content posting and engagement tracking.

[saurabh-khanna]

Data To Request from Instagram

1. Visibility Restriction Details:

   • Data: Logs or metrics on delisting, ranking reductions, or exclusion decisions.
   • Request: Visibility restriction logs tied to hashtags, content types, or account features.

2. Policy Enforcement Metrics:

   • Data: Frequency and type of visibility restrictions applied to posts/accounts.
   • Request: Disaggregated data on enforcement actions by policy type (e.g., sensitive content).

3. Demographic Data:

   • Data: Breakdown of restricted accounts by follower size, region, or account age.
   • Request: Insights into whether certain user groups face more restrictions.

4. Algorithmic Decisions:

   • Data: Criteria or signals used for flagging content (e.g., keywords, metadata).
   • Request: Transparency into algorithmic thresholds or decision-making processes.

5. Notification Data:

   • Data: Logs of notifications sent to users regarding restrictions.
   • Request: Insights into how and when users are informed about restrictions.

6. Sensitive Content Opt-in Impact:

   • Data: Engagement metrics for sensitive vs. non-sensitive content.
   • Request: Data showing how sensitive content opt-in affects discovery.

7. Content Discovery Data:

   • Data: Rankings and visibility decisions for posts in feeds, explore, and search.
   • Request: Detailed metrics on how content discovery is adjusted by platform policies.

8. Historical Policy Changes:

   • Data: Archived versions of Instagram’s policies and visibility guidelines.
   • Request: Access to historical data detailing when and why policies were updated.

[saurabh-khanna]

Python Tools for Student Researchers

1. API Access:

   • instagrapi for streamlined Instagram Graph API interaction.
   • requests or urllib for making custom API calls.

2. Data Analysis:

   • pandas and numpy for organizing and analyzing collected data.
   • matplotlib and seaborn for visualization.

3. NLP and Metadata Analysis:

   • NLTK and spaCy for processing captions and comments.
   • Transformers for sentiment analysis or topic modeling.

4. Web Scraping (as permitted):

   • beautifulsoup4 for HTML parsing.
   • selenium for dynamic content scraping.

5. Image Analysis:

   • OpenCV or Pillow for extracting insights from image metadata or detecting patterns in flagged content.

[saurabh-khanna]

Broad Next Steps

1. Start collecting public data through the Instagram Graph API or via controlled scraping (where permissible).
2. Combine collected data with structured requests to Instagram for granular information about visibility restrictions, policy enforcement, and algorithmic transparency.
3. Leverage Python tools for scalable data analysis and to build reproducible and transparent workflows.