First, the actual background
iQiyi's VIP video can only be viewed by members, and ordinary users can only watch the first 6 minutes. For example, the URL of Pirates of the Caribbean 5: http://#vfrm=19-9-0-1
How do we watch VIP videos for free? An easy way to do this is to parse the website through a cyclone video VIP. URL: http://api.xfsub.com/
This site provides us with free video parsing. Its general parsing method is:
Http://api.xfsub.com/index.php?url=[play address or video id]
For example, for the embroidered spring knife movie, we only need to enter in the browser address bar:
Http://api.xfsub.com/index.php?url=http://#vfrm=19-9-0-1
This way, we can watch these VIP videos online:
However, this website only provides the function of parsing video online. There is no download interface. If you want to download the video, we can use the web crawler to capture the video and download the video.
Second, the actual upgrade
The analysis method is the same, we use Fiddler to capture the packet:
We can see that there are not many useful requests, we analyze them one by one. Let's first look at the information returned by the first request.
You can see that the first request is a GET request, there is no useful information, continue to look at the next one.
We see that the second GET request address has changed, and in the returned message, we see that this page has executed a POST request. What is the POST request? It is the opposite of the GET request. GET is to get data from the server, and the POST request is to send data to the server. The server then returns the corresponding content according to the parameters of the POST request. This POST request has four parameters, time, key, url, and type. Remember this useful information, we look for this request in the packet capture results to see what this POST request did.
Obviously, this is the POST request we are looking for. We can see the parameters of the POST request and the returned data in json format. The parameters stored in the url are as follows:
Xfsub_api\/url.php?key=02896e4af69fb18f7029b6046d7c718&time=1505724557&url=http%3A%2F%2FFv_19rr7qhfg0.html&type=&xml=1
This information has been escaping, but it doesn't matter. We manually extract it and become the following form:
Xfsub_api/url.php?key=02896e4af69fb18f70129b6046d7c718&time=1505724557&url=http://&type=&xml=1
We already know the domain name of the server that parses the video, and then add the domain name:
Http://api.xfsub.com/xfsub_api\url.php?key=02896e4af69fb18f70129b6046d7c718&time=1505724557&url=http://&type=&xml=1
What is stored there? Will not address the video after parsing? We have a browser to open this address to see:
Sure enough, we can see that the video address is close at hand, the URL is as follows:
http://disp.titan.mgtv.com/vod.do?fmt=4&pno=1121&fid=1FEA2622E0BD9A1CA625FBE9B5A238A6&file=/c1/2017/09/06_0/1FEA2622E0BD9A1CA625FBE9B5A238A6_20170906_1_1_705.mp4
Let's open this video address again:
Hey, we got the cached address of this video on the server. Based on this address, we can easily download the video.
PS: It's important to note that these URLs are time-limited and will expire soon because they contain time information. Therefore, when you analyze, you must open the website according to your own URL results to see the video.
Next, our task is to programmatically implement the steps we have analyzed, and obtain the address where the video is stored according to different video playback addresses.
Now sort out the programming ideas:
Use regular expressions to match information such as key, time, and url.
Obtain a url for storing video information by sending a POST request based on the matched information.
According to this url, get the address where the video is stored.
Download the video based on the final video address.
Third, write code
One caveat when writing code is that we need to use requests.session() to keep our session requests. The simple understanding is that the server will assign you a proof of identity when you first access the server. We need to take this ID card to continue the visit. Without this ID, the server will not let you access. This is also the anti-crawling method of this server, which will verify the identity of the user.
We specialize in cable assembly overmolding. Our customized molded cables are built precisly following customers' spec.Experienced proposal can be offered to customers for evaluation by considering stable quality and competitive price.
Custom cable overmoldings improve the quality, look and functionality of many electrical cables. From battery interconnects and molded plugs to custom molded plugs and many other custom designed assemblies – ETOP is a premier custom cable overmolding supplier to OEMs.
we have internal design, prototype, and manufacturing the widest range of molded harnesses, also a diversified line of strain / flex reliefs and grommets.Molded Cables,Stranded Round Molded Cables,Molded Patch Cord Cable,4 Pole Waterproof Molded Cable,Waterproof Connectors
ETOP WIREHARNESS LIMITED , https://www.etopwireharness.com