wordpress 网站 图标,关键词排名查询工具有哪些,wordpress搭建vip下载站,手把手教做网站首先最好能获取一个免费代理#xff0c;来继续下面的阅读和实验
也可以在本机设置代理#xff0c;具体流程由于比较敏感#xff0c;请自行搜索
代理设置成功后的测试网站是 http://www.httpbin.org/get , 访问该链接可以得到请求相关的信息#xff0c;返回结果中的 ori…首先最好能获取一个免费代理来继续下面的阅读和实验
也可以在本机设置代理具体流程由于比较敏感请自行搜索
代理设置成功后的测试网站是 http://www.httpbin.org/get , 访问该链接可以得到请求相关的信息返回结果中的 origin 字段就是客户端的 IP 我们可以根据它判断代理是否设置成功
urllib 的代理设置 from urllib.error import URLError
from urllib.request import ProxyHandler, build_openerproxy 218.87.205.240:22927
proxy_handles ProxyHandler({http: http:// proxy,https: https:// proxy
})opener build_opener(proxy_handles)
try:response opener.open(https://www.httpbin.org/get)print(response.read().decode(utf-8))
except URLError as e:print(e.reason) 这里需要借助 ProxyHandler 对象设置代理 参数是字典类型的数据键名是协议类型 键值是代理地址注意此处的代理地址前面需要地址前面加上协议 即 http:// 或者 https://, 当请求的链接使用的是 HTTPS 协议时 使用 http 对应的代理地址 当请求链接使用的是 HTTPS 协议时 使用 https 键名对应的地址。
创建完 ProxyHandler 对象之后 调用 build_opener 方法传入该对象 创建一个 Opener 对象赋值为 opener 变量 相当于此对象已经设置好了代理。直接调用 opener 的 open 方法就可以访问该链接了
如果代理需要认证的话只需要修改 proxy 变量就可以了 proxy username:password218.87.205.240:22927 如果是 SOCKS 类型那么可以这样设置 import socks
import socket
from urllib.error import URLError
from urllib import requestsocks.set_default_proxy(socks.SOCKS5, 218.87.205.240, 22927)
socket.socket socks.socksocket
try:response request.urlopen(https://www.httpbin.org/get)print(response.read().decode(utf-8))
except URLError as e:print(e.reason) 这里需要导入一个 socks 模块 安装命令 pip install PySocks
requests 代理设置 import requestsproxy 218.87.205.240:22927
proxies {http: http:// proxy,https: https:// proxy
}
try:response requests.get(https://www.httpbin.org/get, proxiesproxies)print(response.text)
except requests.exceptions.ConnectionError as e:print(e.args) 如果需要认证 proxy username:password218.87.205.240:22927 如果代理类型是 SOCKS import requestsproxy 218.87.205.240:22927
proxies {http: socks5:// proxy,https: socks5:// proxy
}
try:response requests.get(https://www.httpbin.org/get, proxiesproxies)print(response.text)
except requests.exceptions.ConnectionError as e:print(e.args) 另外还有一种设置 SOCKS 代理的方法 即使用 socks 模块需要安装 socks 库 import requests
import socks
import socketsocks.set_default_proxy(socks.SOCKS5, 218.87.205.240, 22927)
socket.socket socks.socksocket
try:response requests.get(https://www.httpbin.org/get, proxiesproxies)print(response.text)
except requests.exceptions.ConnectionError as e:print(e.args) 和上面的效果是一样的
httpx 的代理设置 import httpxproxy 218.87.205.240:22927
proxies {http://: socks5:// proxy,https://: socks5:// proxy
}
try:with httpx.Client(proxiesproxies) as client:response client.get(https://www.httpbin.org/get)print(response.text)
except:print(Error) 如果报错说少库 就按照提示 手动安装一下
如果需要认证 proxy username:password218.87.205.240:22927 对于 SOCKS 代理 需要安装 httpx-socks[asyncio] 库 pip install httpx-socks[asyncio]
同步模式 import httpx
from httpx_socks import SyncProxyTransporttransport SyncProxyTransport.from_url(socks5://218.87.205.240:22927)with httpx.Client(transporttransport) as client:response client.get(https://www.httpbin.org/get)print(response.text) 异步模式 import httpx
import asyncio
from httpx_socks import AsyncProxyTransporttransport AsyncProxyTransport.from_url(socks5://218.87.205.240:22927)async def main():async with httpx.AsyncClient(transporttransport) as client:response await client.get(https://www.httpbin.org/get)print(response.text)if __name__ __main__:asyncio.run(main()) Selenium 代理设置 from selenium import webdriverproxy 218.87.205.240:22927
options webdriver.ChromeOptions()
options.add_argument(--proxy-serverhttp:// proxy)
browser webdriver.Chrome(optionsoptions)
browser.get(https://www.httpbin.org/get)
print(browser.page_source)
browser.close() 如果需要认证 from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import zipfileip 127.0.0.1
port 7890
username foo
password barmanifest_json {version:1.0.0,manifest_version: 2,name:Chrome Proxy,permissions: [proxy,tabs,unlimitedStorage,storage,all_urls,webRequest,webRequestBlocking],background: {scripts: [background.js]}
}background_js
var config {mode: fixed_servers,rules: {singleProxy: {scheme: http,host: %(ip) s,port: %(port) s}}}chrome.proxy.settings.set({value: config, scope: regular}, function() {});function callbackFn(details) {return {authCredentials: {username: %(username) s,password: %(password) s}}
}chrome.webRequest.onAuthRequired.addListener(callbackFn,{urls: [all_urls]},[blocking]
)% {ip: ip, port: port, username: username, password: password}plugin_file proxy_auth_plugin.zip
with zipfile.ZipFile(plugin_file, w) as zp:zp.writestr(manifest.json, manifest_json)zp.writestr(background.js, background_js)
options Options()
options.add_argument(--start-maximized)
options.add_extension(plugin_file)
browser webdriver.Chrome(optionsoptions)
browser.get(https://httpbin.org/get)
print(browser.page_source)
browser.close() 这里在本地创建了一个 manifest.json 配置文件 和 backgound.js 脚本来设置认证代理运行代码后 本地会生成一个 proxy_auth_plugin.zip 文件来保存当前配置
SOCKS 代理设置 from selenium import webdriverproxy 127.0.0.1:7891
options webdriver.ChromeOptions()
options.add_argument(--proxy-serversocks5:// proxy)
browser webdriver.Chrome(optionsoptions)
browser.get(https://httpbin.org/get)
print(browser.page_source)
browser.close() aiohttp 的代理设置 import asyncio
import aiohttpproxy http://127.0.0.1:7890async def main():async with aiohttp.ClientSession() as session:async with session.get(https://httpbin.org/get, proxyproxy) as response:print(await response.text())if __name__ __main__:asyncio.get_event_loop().run_until_complete(main()) 如果需要认证 proxy http://username:passwordhttp://127.0.0.1:7890 SOCKS 代理设置 需要安装一个支持库 pip install aiohttp-socks import asyncio
import aiohttp
from aiohttp_socks import ProxyConnector, ProxyType# connector ProxyConnector.from_url(socks5://127.0.0.1:7891)connector ProxyConnector(proxy_typeProxyType.HTTP,host127.0.0.1,port7890,# usernameuser,# passwordpassword,# rdnsTrue
)async def main():async with aiohttp.ClientSession(connectorconnector) as session:async with session.get(https://httpbin.org/get) as response:print(await response.text())if __name__ __main__:asyncio.get_event_loop().run_until_complete(main()) Pyppeteer 的代理设置
对于 pyppeteer 默认的是 Chrome 的 Chromium 浏览器 import asyncio
from pyppeteer import launchproxy 127.0.0.1:7890async def main():browser await launch({args: [--proxy-serverhttp:// proxy], headless: False})page await browser.newPage()await page.goto(https://httpbin.org/get)print(await page.content())await browser.close()if __name__ __main__:asyncio.get_event_loop().run_until_complete(main()) SOCKS 代理设置 import asyncio
from pyppeteer import launchproxy 127.0.0.1:7891async def main():browser await launch({args: [--proxy-serversocks5:// proxy], headless: False})page await browser.newPage()await page.goto(https://httpbin.org/get)print(await page.content())await browser.close()if __name__ __main__:asyncio.get_event_loop().run_until_complete(main()) PlayWright 的代理设置 from playwright.sync_api import sync_playwrightwith sync_playwright() as p:browser p.chromium.launch(headlessFalse, proxy{server: http://127.0.0.1:7890})page browser.new_page()page.goto(https://httpbin.org/get)print(page.content())browser.close() 在调用 launch 方法的时候 可以传入 proxy 参数 它是一个字典 其中有一个必填的字段叫作 server 这里我们直接填入 HTTP 代理的地址即可
需要认证的时候 from playwright.sync_api import sync_playwrightwith sync_playwright() as p:browser p.chromium.launch(proxy{server: http://127.0.0.1:7890,username: foo,password: bar})page browser.new_page()page.goto(https://httpbin.org/get)print(page.content())browser.close() SOCKS 代理 from playwright.sync_api import sync_playwrightwith sync_playwright() as p:browser p.chromium.launch(proxy{server: socks5://127.0.0.1:7891})page browser.new_page()page.goto(https://httpbin.org/get)print(page.content())browser.close()
