• Home
  • Event
  • Article
    • Gallery
    • Opinion
    • Resources
  • About
    • Founding Members
    • Steering Committee
    • Executive Officers
  • Newsletter

The Data & News Society

~ news/numbers; stats/stories

The Data & News Society

Tag Archives: Scraping

Some Scraping Targets and Ideas

10 Saturday Mar 2018

Posted by Pili Hu in Resources

≈ Leave a comment

Tags

COMM7780/JOUR7280, data collection, Scraping

This is a casual post to dump some target sites for scraping or just project ideas. Those messages were first sent through COMM7780/JOUR7280 WeChat group. Although we have only explored part of those possibilities this semester, the list is good for future reference. We can bounce off ideas in the comment below and enrich this list.

Top Targets: Movie, Shopping and News

Let’s first have a look at what the students care about from HW2 submission:

Scraping Targets from HW2 submission – COMM7780/JOUR7280

Continue reading →

網絡數據包分析:從 Google Maps 獲取 Fusion Table 原始數據

26 Sunday Nov 2017

Posted by Pili Hu in Tool

≈ Leave a comment

Tags

data collection, Network Analysis, Scraping

網上經常見到使用 Google Maps 繪製的地圖,如果希望對地圖中的興趣點(Point of Interest,POI)進行二次分析,就需要得到繪製地圖背後的結構化數據。如果是使用 Google Fusion Table 繪製的地圖,可以通過網絡抓包找到 Fusion Table 的ID,進而拼接出原始地址。本文來自同學 Lam Man Kit 的投稿,僅做技術交流。數據記者在使用時,需要注意原始數據的版權。而本地的研究者也需要遵守公平使用原則。本文以 FactWire 的數據報道 「分析182個領展停車場月租收費 9成2貴過房委會同區 最大差距達1.18倍」 爲例。

圖:通過網絡抓包分析 Fusion Table 的 ID

Continue reading →

Top Posts & Pages

  • Create Simple Filled Map (HK) in Tableau
  • Google Sheets 技巧總結
  • Analyze tracking errors between ETF and stock market index in the last decade
  • The Setup of D&N Society
  • Abortion in China: policy and data

Recent Posts

  • A dossier of data journalism teaching strategies: Words from journalism educators worldwide
  • “中国数据可视化大赛”创作者专访:数据“几人行”
  • “Whoah, wait a minute, every reporter needs to be a data reporter”: Conversations with two generations of data journalists at the Los Angeles Times
  • Aaron Mendelson: Would numbers work with radio?
  • 首届“中国数据可视化大赛”启动-数据中的宏观和微观世界
  • Job Opportunity: Market Information Specialist at Unum Networks

Recent Comments

A quick video I made… on New towns fail to be self-cont…
Erin Chan on Create Simple Filled Map (HK)…
National Congress: s… on “Big Data” Tells Y…
Pili Hu on Data News of the Week | Gender…
Pili Hu on Key Takes from Jessica Lo…

Archives

  • August 2019
  • June 2019
  • April 2019
  • March 2019
  • February 2019
  • December 2018
  • November 2018
  • October 2018
  • September 2018
  • June 2018
  • May 2018
  • April 2018
  • March 2018
  • February 2018
  • January 2018
  • December 2017
  • November 2017
  • October 2017
  • September 2017
  • August 2017
  • May 2017
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • September 2016
  • August 2016
  • July 2016
  • June 2016
  • May 2016
  • April 2016
  • March 2016
  • February 2016
  • January 2016
  • December 2015
  • November 2015

Meta

  • Register
  • Log in
  • Entries feed
  • Comments feed
  • WordPress.com

Recent Posts

  • A dossier of data journalism teaching strategies: Words from journalism educators worldwide
  • “中国数据可视化大赛”创作者专访:数据“几人行”
  • “Whoah, wait a minute, every reporter needs to be a data reporter”: Conversations with two generations of data journalists at the Los Angeles Times
  • Aaron Mendelson: Would numbers work with radio?
  • 首届“中国数据可视化大赛”启动-数据中的宏观和微观世界

Recent Comments

A quick video I made… on New towns fail to be self-cont…
Erin Chan on Create Simple Filled Map (HK)…
National Congress: s… on “Big Data” Tells Y…
Pili Hu on Data News of the Week | Gender…
Pili Hu on Key Takes from Jessica Lo…

Archives

  • August 2019
  • June 2019
  • April 2019
  • March 2019
  • February 2019
  • December 2018
  • November 2018
  • October 2018
  • September 2018
  • June 2018
  • May 2018
  • April 2018
  • March 2018
  • February 2018
  • January 2018
  • December 2017
  • November 2017
  • October 2017
  • September 2017
  • August 2017
  • May 2017
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • September 2016
  • August 2016
  • July 2016
  • June 2016
  • May 2016
  • April 2016
  • March 2016
  • February 2016
  • January 2016
  • December 2015
  • November 2015

Categories

  • Announcement
  • Announcements
  • Article
  • Book
  • Colloquium
  • comment
  • Event
  • Field Trip
  • Gallery
  • general
  • news story
  • Open Lecture
  • Opinion
  • Resources
  • Tool
  • Tutorial
  • Uncategorized

Meta

  • Register
  • Log in
  • Entries feed
  • Comments feed
  • WordPress.com

Recent Posts

  • A dossier of data journalism teaching strategies: Words from journalism educators worldwide
  • “中国数据可视化大赛”创作者专访:数据“几人行”
  • “Whoah, wait a minute, every reporter needs to be a data reporter”: Conversations with two generations of data journalists at the Los Angeles Times
  • Aaron Mendelson: Would numbers work with radio?
  • 首届“中国数据可视化大赛”启动-数据中的宏观和微观世界

Recent Comments

A quick video I made… on New towns fail to be self-cont…
Erin Chan on Create Simple Filled Map (HK)…
National Congress: s… on “Big Data” Tells Y…
Pili Hu on Data News of the Week | Gender…
Pili Hu on Key Takes from Jessica Lo…

Archives

  • August 2019
  • June 2019
  • April 2019
  • March 2019
  • February 2019
  • December 2018
  • November 2018
  • October 2018
  • September 2018
  • June 2018
  • May 2018
  • April 2018
  • March 2018
  • February 2018
  • January 2018
  • December 2017
  • November 2017
  • October 2017
  • September 2017
  • August 2017
  • May 2017
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • September 2016
  • August 2016
  • July 2016
  • June 2016
  • May 2016
  • April 2016
  • March 2016
  • February 2016
  • January 2016
  • December 2015
  • November 2015

Categories

  • Announcement
  • Announcements
  • Article
  • Book
  • Colloquium
  • comment
  • Event
  • Field Trip
  • Gallery
  • general
  • news story
  • Open Lecture
  • Opinion
  • Resources
  • Tool
  • Tutorial
  • Uncategorized

Meta

  • Register
  • Log in
  • Entries feed
  • Comments feed
  • WordPress.com

Blog at WordPress.com.

  • Follow Following
    • The Data & News Society
    • Already have a WordPress.com account? Log in now.
    • The Data & News Society
    • Customize
    • Follow Following
    • Sign up
    • Log in
    • Report this content
    • View site in Reader
    • Manage subscriptions
    • Collapse this bar
 

Loading Comments...