Lab: Semantic Lifting - HTML - Revision history

Sinoa at 07:57, 6 April 2021

2021-04-06T07:57:56Z

2020-03-27T09:42:09Z

← Older revision		Revision as of 09:42, 27 March 2020
Line 63:		Line 63:
	</syntaxhighlight>		</syntaxhighlight>

			You can use this regex to extract the id from the Corpus ID, or the topic ID (which is in the URL)
			<syntaxhighlight>
			id = re.findall('\d+$', id)[0]
			</syntaxhighlight>

	==Task 2==		==Task 2==
Line 110:		Line 114:

	</syntaxhighlight>		</syntaxhighlight>


	~~{\| role="presentation" class="wikitable mw-collapsible mw-collapsed"~~
	~~\| <strong>Hints</strong>~~
	\|-
	\|
	\|}


	==Useful Reading==		==Useful Reading==
	* [https://www.dataquest.io/blog/web-scraping-tutorial-python/ Dataquest.io - Web-scraping with Python]		* [https://www.dataquest.io/blog/web-scraping-tutorial-python/ Dataquest.io - Web-scraping with Python]

2020-03-27T09:36:32Z

← Older revision		Revision as of 09:36, 27 March 2020
Line 106:		Line 106:
	for paper in papers:		for paper in papers:
	# e.g selecting title.		# e.g selecting title.
	title = paper.find('div', attrs={'class': 'timeline-paper-title'})		title = paper.find('div', attrs={'class': 'timeline-paper-title'}).text
	print(title~~.text~~)		print(title)

	</syntaxhighlight>		</syntaxhighlight>

2020-03-27T09:31:11Z

← Older revision		Revision as of 09:31, 27 March 2020
Line 101:		Line 101:

	# Find the html that surrounds all the papers		# Find the html that surrounds all the papers
	papers = html~~.body~~.find_all('div', attrs={'class': 'flex-container'})		papers = html.find_all('div', attrs={'class': 'flex-container'})

	# Iterate through each paper to make triples:		# Iterate through each paper to make triples:

2020-03-27T09:25:26Z

← Older revision		Revision as of 09:25, 27 March 2020
Line 50:		Line 50:
	Knowing this we can use BeautifulSoup to find this in python code e.g:		Knowing this we can use BeautifulSoup to find this in python code e.g:
	<syntaxhighlight>		<syntaxhighlight>
	topic = html~~.body~~.find('h1', attrs={'class': 'entity-name'}).text		topic = html.find('h1', attrs={'class': 'entity-name'}).text
	</syntaxhighlight>		</syntaxhighlight>

Line 56:		Line 56:

	<syntaxhighlight>		<syntaxhighlight>
	papers = html~~.body~~.find_all('div', attrs={'class': 'flex-container'})		papers = html.find_all('div', attrs={'class': 'flex-container'})
	for paper in papers:		for paper in papers:
	# e.g selecting title.		# e.g selecting title.

2020-03-27T09:13:29Z

← Older revision		Revision as of 09:13, 27 March 2020
Line 67:		Line 67:
	Create triples for the Topic of the page ("Knowledge Graph").		Create triples for the Topic of the page ("Knowledge Graph").

	For example, a topic as related topics (on the top-right of the page). It also has, "known as" values, and a description.		For example, a topic has related topics (on the top-right of the page). It also has, "known as" values, and a description.

	This is a good opportunity to use the SKOS vocabulary to describe Concepts.		This is a good opportunity to use the SKOS vocabulary to describe Concepts.

2020-03-27T08:56:39Z

2020-03-27T08:56:29Z

← Older revision		Revision as of 08:56, 27 March 2020
Line 87:		Line 87:
	from rdflib import Graph, Literal, URIRef, Namespace		from rdflib import Graph, Literal, URIRef, Namespace
	from rdflib.namespace import RDF, OWL, SKOS, RDFS, XSD		from rdflib.namespace import RDF, OWL, SKOS, RDFS, XSD

	import requests		import requests
	import re		import re

2020-03-27T08:45:47Z

← Older revision		Revision as of 08:45, 27 March 2020
Line 86:		Line 86:
	from bs4 import BeautifulSoup as bs		from bs4 import BeautifulSoup as bs
	from rdflib import Graph, Literal, URIRef, Namespace		from rdflib import Graph, Literal, URIRef, Namespace
	from rdflib.namespace import RDF, OWL, SKOS		from rdflib.namespace import RDF, OWL, SKOS, RDFS, XSD

	import requests		import requests
	import re		import re

2020-03-27T08:24:26Z