Skip to content

Conversation

@alsrua7222
Copy link

http_request_randomizer.requests.errors.ProxyListException.ProxyListException: list is empty
-> FreeProxyParser.py: html parsing is list error. (now solved.)

reqeusts.exceptions.InvalidURL: Failed to parse: http://(ip):(port) | FreeProxy
-> UnPack.py: headers add
-> PremProxyParser.py: headers add

@pgaref
Copy link
Owner

pgaref commented Jan 9, 2022

Thanks for the PR @alsrua7222 !
Tested locally and fixes both PremProxyParser by adding headers on the requests as #74 and FreeProxyParser by updating the attributes.
Can you please fix test_FreeProxyParser so we can merge this? I believe a simple update on free_proxy_mock is needed.

@urlmatch(netloc=r'(.*\.)?free-proxy-list\.net$') def free_proxy_mock(url, request): return """<table border="0" cellpadding="0" cellspacing="0" class="table table-striped table-bordered"

@pgaref pgaref changed the title solution work for FreeProxy FIX PremProxy and FreeProxy Jan 9, 2022
@pgaref pgaref changed the title FIX PremProxy and FreeProxy FIX for PremProxy and FreeProxy Jan 9, 2022
@alsrua7222
Copy link
Author

I didn't know if it was updated or not.
It's already been updated.
I wasted my time.
However, libraries help a lot with proxies. thank you.
After an error, when I traced back, the html parsing structure was changed.
Also, I wanted to point out that "headers" should be added to the library.

As you said, we simply updated the html table structure of free_proxy_list.net.
Is it correct to do this?

@pgaref
Copy link
Owner

pgaref commented Jan 12, 2022

I didn't know if it was updated or not. It's already been updated. I wasted my time. However, libraries help a lot with proxies. thank you. After an error, when I traced back, the html parsing structure was changed. Also, I wanted to point out that "headers" should be added to the library.

As you said, we simply updated the html table structure of free_proxy_list.net. Is it correct to do this?

Hey @alsrua7222 thanks for updating the PR!
The new html table structure is fine but since you used different proxy exmamples the assertion of the Test should also be fixed.

Please take a look at:

  self.assertEqual(proxy_list_addr, free_proxy_expected)

E AssertionError: Lists differ: ['58.234.116.197:8193', '20.122.24.225:80',[40 chars]080'] != ['138.197.136.46:3128', '177.207.75.227:8080']
E
E First differing element 0:
E '58.234.116.197:8193'
E '138.197.136.46:3128'
E
E First list contains 2 additional elements.
E First extra element 2:
E '154.236.177.100:1981'
E
E + ['138.197.136.46:3128', '177.207.75.227:8080']
E - ['58.234.116.197:8193',
E - '20.122.24.225:80',
E - '154.236.177.100:1981',
E - '54.37.160.92:1080']

@alsrua7222 alsrua7222 closed this Jan 12, 2022
@pgaref pgaref reopened this Jan 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants