1.簡介
jsoup is a Java library for working with real-world HTML. It provides a very convenient API for fetching URLs and extracting and manipulating data, using the best of HTML5 DOM methods and CSS selectors.
jsoup是一個用於處理實際HTML的Java庫。它使用HTML5最佳DOM方法和CSS選擇器，爲提取URL以及提取和處理數據提供了非常方便的API。
2.使用
2.1 導入依賴
implementation 'org.jsoup:jsoup:1.13.1'
2.2 解析字符串內容
String html = "<html><head><title>First parse</title></head>"
+ "<body><p>Parsed HTML into a doc.</p></body></html>";
Document doc = Jsoup.parse(html);
2.3 解析頁面內容的一部分內容
String html = "<div><p>Lorem ipsum.</p>";
Document doc = Jsoup.parseBodyFragment(html);
Element body = doc.body();
2.4 從網頁加載
Document doc = Jsoup.connect("http://example.com/").get();
String title = doc.title();
也可以執行post
Document doc = Jsoup.connect("http://example.com")
.data("query", "Java")
.userAgent("Mozilla")
.cookie("auth", "token")
.timeout(3000)
.post();
2.5 從文件加載內容
File input = new File("/tmp/input.html");
Document doc = Jsoup.parse(input, "UTF-8", "http://example.com/");
3.提取數據
3.1 使用Dom方法瀏覽文檔
File input = new File("/tmp/input.html");
Document doc = Jsoup.parse(input, "UTF-8", "http://example.com/");
Element content = doc.getElementById("content");
Elements links = content.getElementsByTag("a");
for (Element link : links) {
String linkHref = link.attr("href");
String linkText = link.text();
}
3.2 使用選擇器 查找元素
File input = new File("/tmp/input.html");
Document doc = Jsoup.parse(input, "UTF-8", "http://example.com/");
//有鏈接的a標籤
Elements links = doc.select("a[href]"); // a with href
//以.png結束的img
Elements pngs = doc.select("img[src$=.png]");
// img with src ending .png
//以class名爲mastthead的div
Element masthead = doc.select("div.masthead").first();
// div with class=masthead
//h3 後a 標籤
Elements resultLinks = doc.select("h3.r > a"); // direct a after h3
4 案例
List<TopicListBean> mList = new ArrayList<>();
Elements itemElements = doc.select("div.cell.item"); //item根節點
int count = itemElements.size();
for (int i = 0; i < count; i++) {
Elements titleElements = itemElements.get(i).select("div.cell.item table tr td span.item_title > a"); //標題
Elements imgElements = itemElements.get(i).select("div.cell.item table tr td img.avatar"); //頭像
Elements commentElements = itemElements.get(i).select("div.cell.item table tr a.count_livid"); //評論數
Elements nodeElements = itemElements.get(i).select("div.cell.item table tr span.small.fade a.node"); //節點
Elements nameElements = itemElements.get(i).select("div.cell.item table tr span.small.fade strong a"); //作者 & 最後回覆
Elements timeElements = itemElements.get(i).select("div.cell.item table tr span.small.fade"); //更新時間
TopicListBean bean = new TopicListBean();
if (titleElements.size() > 0) {
bean.setTitle(titleElements.get(0).text());
bean.setTopicId(parseId(titleElements.get(0).attr("href")));
}
if (imgElements.size() > 0) {
bean.setImgUrl(parseImg(imgElements.get(0).attr("src"))); // http:
}
if (nodeElements.size() > 0) {
bean.setNode(nodeElements.get(0).text());
}
if (nameElements.size() > 0) {
bean.setName(nameElements.get(0).text());
}
//存在沒有 最後回覆者、評論數、更新時間的情況
if (nameElements.size() > 1) {
bean.setLastUser(nameElements.get(1).text());
}
if (commentElements.size() > 0) {
bean.setCommentNum(Integer.valueOf(commentElements.get(0).text()));
}
if (timeElements.size() > 1) {
bean.setUpdateTime(parseTime(timeElements.get(1).text()));
}
mList.add(bean);
}