项目作者: tim232385

项目描述 :
Web crawler.
高级语言: Java
项目地址: git://github.com/tim232385/WebVideoBot.git
创建时间: 2018-10-29T17:28:48Z
项目社区:https://github.com/tim232385/WebVideoBot

开源协议:MIT License

下载




Web Vedeo Bot

java-1.8
license: MIT

  • Easy install, all dependencies are already included, just need to have a java environment.

porn_flow

Environment, Architecture

  • Java1.8

  • Crawler4j

  • Spring Boot x H2 Db

How to use

git clone https://github.com/tim232385/WebVideoBot.git
cd WebVideoBot
java -jar PornBot.jar
**DOWNLOAD_VIDEO default is N** See Default Configuration

Demo
h2_console

Database Description

http://localhost:8000/h2-console/

JDBC URL: jdbc:h2:~/porn/porn-db

User Name: sa

Password: empty

Record Table:

  1. Table_Name PORN_RECORD
  2. viewKey The website's video unique key.
  3. imageUrl :Image url of video.
  4. linkUrl :Video jump to Website`s link
  5. videoUrl :Video adrress.
  6. videoTitle :Title of video.
  7. videoDuration :Video click count.
  8. videoQuality :Defualt quality - 240, 480, 960, 1280p.
  9. download :Has been downloaded. True or false.
  10. createdTime :The record created time.
  11. filePath :The video downloaded path.

Configuration

  1. FILE_PATH Video download path.
  2. VIDEO_DOWNLOAD_SIZE Maximum download size of the video.(Byte)
  3. MAX_PAGE_SIZE Crawling page size.
  4. CONCURRENT_THREAD_SIZE Muilti-threads request amount.
  5. START_URL Crawling url.
  6. DOWNLOAD_VIDEO Download video. Y or N.

[See Default Configuration](https://github.com/tim232385/PornBot/blob/master/config.properties

Stargazers over time

Stargazers over time