Label which links are duplicate / remove duplicate links

[9mido]

Some links belong in multiple categories which is how things are currently set up. If a link is repeated somewhere else, label it as a duplicate to avoid people from re-reading the same thing again. Or simply, just put next to each duplicate link something like "this link shows up in multiple categories", "duplicate", " * ", color code them, or whatever you think is easier.

Or instead of having category headers and the links below them, just have all the links shown without headers and next to each link say which categories they belong to and do not repeat any of the links. This will be better for SEO.

https://www.siteliner.com/endthefud.org?siteliner=site-dashboard&siteliner-sort=scan_time&siteliner-from=1&siteliner-message=

There are also quite a few links that bring you to the same URL but have different anchor tags referencing that particular category. I don't think this is necessary. https://safehodl.github.io/failure is an example of this.

Find all URLs and sort them:

wget -q https://endthefud.org/ -O - | tr "\t\r\n'" ' "' | grep -i -o '<a[^>]\+href[ ]*=[ \t]*"\(ht\|f\)tps\?:[^"]\+"' | sed -e 's/^.*"\([^"]\+\)".*$/\1/g' | sort

Output:

http://btcco2.com
http://btcco2.com
http://btcco2.org
http://btcco2.org
https://allenfarrington.medium.com/
https://arxiv.org/abs/1710.10377
https://bitcoinandenergy.medium.com/bitcoin-mining-and-the-case-for-more-energy-90094ce25fac
https://bitcoinmagazine.com/business/bitcoin-energy-use-compare-industry
https://bitcoinmagazine.com/business/op-ed-bitcoin-miners-consume-reasonable-amount-energy-and-its-all-worth-it
https://bitcoinmagazine.com/culture/bitcoin-vs-world-military-emissions
https://bitcoinmagazine.com/culture/the-hidden-costs-of-the-petrodollar
https://bitcoin.stackexchange.com/questions/6062/what-effects-would-a-scalable-quantum-computer-have-on-bitcoin
https://blog.coinbase.com/fact-check-is-bitcoin-mining-environmentally-unfriendly-3559823af6f1
https://blog.getadmiral.com/registration-wall
https://blog.lopp.net/are-chinese-miners-threat-bitcoin/
https://blog.lopp.net/are-chinese-miners-threat-bitcoin/
https://blog.lopp.net/who-controls-bitcoin-core-/
https://blog.trezor.io/three-myths-about-bitcoins-energy-consumption-ef613a1f3d5
https://casebitcoin.com/critiques
https://casebitcoin.com/critiques/bitcoin-can-be-cloned
https://casebitcoin.com/critiques/bitcoin-gets-hacked
https://casebitcoin.com/critiques/bitcoin-is-a-bubble
https://casebitcoin.com/critiques/bitcoin-is-pumped-by-tether
https://casebitcoin.com/critiques/bitcoin-is-used-by-criminals
https://casebitcoin.com/critiques/bitcoin-wastes-energy
https://casebitcoin.com/critiques/china-controls-bitcoin
https://casebitcoin.com/critiques/gov-will-shut-it-down
https://casebitcoin.com/critiques/no-intrinsic-value
https://coinshares.com/research/bitcoin-mining-network-december-2019
https://danhedl.medium.com/bitcoins-distribution-was-fair-e2ef7bbbc892
https://danhedl.medium.com/pow-is-efficient-aa3d442754d3
https://danheld.substack.com/p/dont-fear-tether
https://dergigi.com/2018/06/10/bitcoin-s-energy-consumption/
https://dergigi.com/2021/08/02/implications-of-outlawing-bitcoin/
https://dergigi.com/bitcoin/
https://docsend.com/view/adwmdeeyfvqwecj2
https://download.wpsoftware.net/bitcoin/pos.pdf
https://endthefud.org/altcoins
https://endthefud.org/ban
https://endthefud.org/china
https://endthefud.org/control
https://endthefud.org/criminals
https://endthefud.org/energy
https://endthefud.org/governments
https://endthefud.org/inflation
https://endthefud.org/justice
https://endthefud.org/miners
https://endthefud.org/money
https://endthefud.org/ponzi
https://endthefud.org/PoS
https://endthefud.org/PoW
https://endthefud.org/tech
https://en.m.wikipedia.org/wiki/Subjective_theory_of_value
https://github.com/libbitcoin/libbitcoin-system/wiki/ASIC-Monopoly-Fallacy
https://github.com/libbitcoin/libbitcoin-system/wiki/Cryptoeconomics
https://github.com/libbitcoin/libbitcoin-system/wiki/Energy-Exhaustion-Fallacy
https://github.com/libbitcoin/libbitcoin-system/wiki/Energy-Waste-Fallacy
https://github.com/libbitcoin/libbitcoin-system/wiki/Proof-of-Stake-Fallacy
https://github.com/libbitcoin/libbitcoin-system/wiki/Regression-Fallacy
https://github.com/libbitcoin/libbitcoin-system/wiki/Shitcoin-Definition
https://github.com/libbitcoin/libbitcoin-system/wiki/Value-Proposition
https://github.com/pox/EndTheFUD
https://github.com/pox/EndTheFUD
https://hackernoon.com/the-bitcoin-vs-visa-electricity-consumption-fallacy-8cf194987a50
https://hassmccook.medium.com/comparing-bitcoins-environmental-impact-f56b18014f64
https://hbr.org/2021/05/how-much-energy-does-bitcoin-actually-consume
http://shitcoin.review
http://shitcoin.review
https://hugonguyen.medium.com/
https://hugonguyen.medium.com/proof-of-stake-the-wrong-engineering-mindset-15e641ab65a2
https://hugonguyen.medium.com/work-is-timeless-stake-is-not-554c4450ce18
https://journal.bitcoinreserve.com/bitcoin-a-bold-american-future/
https://mdotbit.medium.com/addressing-concerns-about-bitcoins-electricity-use-378e0de4af42
https://medium.com/@BobMcElrath/whats-wrong-with-proof-of-stake-77d4f370be15
https://medium.com/coinshares/beware-of-lazy-research-c828c900b7d5
https://medium.com/hackernoon/asic-resistance-is-nothing-but-a-blockchain-buzzword-b91d3d770366
https://medium.com/hackernoon/betterhash-decentralizing-bitcoin-mining-with-new-hashing-protocols-291de178e3e0
https://medium.com/hackernoon/bitcoin-miners-beware-invalid-blocks-need-not-apply-51c293ee278b
https://medium.com/hackernoon/green-energy-and-bitcoin-eb7961f38382
https://medium.com/john-pfeffer/an-institutional-investors-take-on-cryptoassets-690421158904
https://medium.com/@knut.svanholm
https://medium.com/@mukeat/carbon-footprint-differences-btc-vsbanks-a5f8f0b844d0
https://medium.com/@nic__carter
https://medium.com/@nic__carter/a-most-peaceful-revolution-8b63b64c203e
https://medium.com/@nic__carter/in-support-of-the-proof-of-work-un-fair-launch-cd6e8f06358f
https://medium.com/@nic__carter/its-the-settlement-assurances-stupid-5dcd1c3f4e41
https://medium.com/@nic__carter/noahbjectivity-on-bitcoin-mining-2052226310cb
https://medium.com/@nic__carter/on-bitcoin-the-gray-lady-embraces-climate-lysenkoism-a2d31e465ec0
https://medium.com/stakin/centralization-of-stake-in-pos-f7ccb8f8254
https://mises.org/library/inflation-and-deflation
https://nakamotoinstitute.org/literature/
https://nakamotoinstitute.org/mempool/bitcoin-cannot-be-banned/
https://nakamotoinstitute.org/mempool/bitcoin-cant-be-copied/
https://nakamotoinstitute.org/mempool/bitcoin-does-not-waste-energy/
https://nakamotoinstitute.org/mempool/bitcoin-is-not-a-pyramid-scheme/
https://nakamotoinstitute.org/mempool/bitcoin-is-not-backed-by-nothing/
https://nakamotoinstitute.org/mempool/bitcoin-is-not-for-criminals/
https://nakamotoinstitute.org/mempool/bitcoin-is-not-too-slow/
https://nakamotoinstitute.org/mempool/bitcoin-is-not-too-volatile/
https://nakamotoinstitute.org/mempool/bitcoin-is-the-great-definancialization/
https://nakamotoinstitute.org/mempool/bitcoin-not-blockchain/
https://nakamotoinstitute.org/mempool/bitcoin-obsoletes-all-other-money/
https://nakamotoinstitute.org/mempool/ethereum-is-doomed/
https://nakamotoinstitute.org/mempool/how-we-know-bitcoin-is-not-a-bubble/
https://nakamotoinstitute.org/mempool/speculative-attack/
https://nakamotoinstitute.org/mempool/the-bitcoin-central-banks-perfect-monetary-policy/
https://nakamotoinstitute.org/mempool/the-bullish-case-for-bitcoin/
https://nakamotoinstitute.org/shelling-out/
https://nymag.com/intelligencer/2021/05/jack-dorsey-says-bitcoin-is-climate-friendly-is-he-right.html
https://quillette.com/2021/02/21/can-governments-stop-bitcoin/
https://safehodl.github.io/failure
https://safehodl.github.io/failure/#block-rewards-will-stop
https://safehodl.github.io/failure/#can-be-forked-infinitely
https://safehodl.github.io/failure/#central-bank-digital-currencies
https://safehodl.github.io/failure/#disinflationary-money-breaks-the
https://safehodl.github.io/failure/#governments-will-stop
https://safehodl.github.io/failure/#miners-are-centralized-and
https://safehodl.github.io/failure/#quantum-computing-will-break
https://safehodl.github.io/failure/#scarcity-can-be-changed
https://saifedean.com/book/
https://saifedean.com/book/
https://timevalueofbtc.medium.com/
https://tomerstrolight.medium.com/
https://tomerstrolight.medium.com/bitcoins-fundamental-value-is-that-it-is-the-most-excellent-money-that-exists-24be5e1d6b98
https://tomerstrolight.medium.com/the-problem-with-ethereum-af9692f4af95
https://tomerstrolight.medium.com/why-bitcoins-imitators-are-scams-e38fab4c78ba
https://tomerstrolight.medium.com/why-bitcoin-the-series-660fe20ec244
https://tonevays.com/
https://tonevays.com/blog/what-is-crypto-decentralization-and-why-bitcoin-stands-alone
https://unchained-capital.com/blog/author/plewis/
https://unchained-capital.com/blog/bitcoin-is-antifragile/
https://voskuil.org/cryptoeconomics/
https://www.amazon.com/Thank-God-Bitcoin-Corruption-Redemption/dp/1641991216/ref=sr_1_1?dchild=1&keywords=thank+god+for+bitcoin&qid=1607152012&sr=8-1
https://www.citadel21.com/bitcoin-is-the-first-global-market-for-electricity-and-will-unleash-renewables
https://www.coincenter.org/how-bitcoin-could-drive-the-clean-energy-revolution/
https://www.coindesk.com/bitcoin-mining-wastes-energy-thats-good-thing
https://www.coindesk.com/frustrating-maddening-all-consuming-bitcoin-energy-debate
https://www.coindesk.com/the-last-word-on-bitcoins-energy-consumption
https://www.coindesk.com/what-bloomberg-gets-wrong-about-bitcoins-climate-footprint
https://www.fidelitydigitalassets.com/articles/addressing-bitcoin-criticisms
https://www.fidelitydigitalassets.com/articles/addressing-bitcoin-criticisms
https://www.fidelitydigitalassets.com/articles/addressing-bitcoin-criticisms
https://www.fidelitydigitalassets.com/articles/addressing-bitcoin-criticisms
https://www.fidelitydigitalassets.com/articles/addressing-bitcoin-criticisms
https://www.forbes.com/sites/rogerhuang/2020/12/21/heres-why-quantum-computing-will-not-break-cryptocurrencies/?sh=24a95b3c167b
https://www.lopp.net/bitcoin-information/getting-started.html
https://www.lopp.net/bitcoin-information/getting-started.html#misconceptions
https://www.lynalden.com/author/penumbra/
https://www.lynalden.com/bitcoin-ponzi-scheme/
https://www.lynalden.com/bitcoin-security-modeling/
https://www.lynalden.com/bitcoins-network-effect/
https://www.lynalden.com/ethereum-analysis/
https://www.lynalden.com/fraying-petrodollar-system/
https://www.lynalden.com/misconceptions-about-bitcoin/
https://www.lynalden.com/misconceptions-about-bitcoin/
https://www.lynalden.com/misconceptions-about-bitcoin/
https://www.lynalden.com/misconceptions-about-bitcoin/
https://www.pairagraph.com/dialogue/9f0dfc47978740ebb0439efaadf23f41/2
https://www.swanbitcoin.com/bitcoins-energy-usage-is-not-a-problem-heres-why-by-lyn-alden/
https://www.theblockcrypto.com/post/11481/asic-chip-technology-is-renewable-energys-future
https://www.truthcoin.info/blog/measuring-decentralization/
https://www.truthcoin.info/blog/pos-still-pointless/
https://www.truthcoin.info/blog/pow-and-mining
https://www.truthcoin.info/blog/pow-cheapest/
https://www.upstreamdata.ca/post/natural-gas-venting-how-bitcoin-solved-a-160-year-old-problem
https://youtu.be/ns_I2LpeAbQ?t=26084

Find only duplicates:

wget -q https://endthefud.org/ -O - | tr "\t\r\n'" ' "' | grep -i -o '<a[^>]\+href[ ]*=[ \t]*"\(ht\|f\)tps\?:[^"]\+"' | sed -e 's/^.*"\([^"]\+\)".*$/\1/g' | sort | uniq -d

Output:

http://btcco2.com
http://btcco2.org
https://blog.lopp.net/are-chinese-miners-threat-bitcoin/
https://github.com/pox/EndTheFUD
http://shitcoin.review
https://saifedean.com/book/
https://www.fidelitydigitalassets.com/articles/addressing-bitcoin-criticisms
https://www.lynalden.com/misconceptions-about-bitcoin/

[pox]

@9mido looking at the articles in the output above, AFAICS they all appear with the same exactly link when they appear multiple times, which is fine (browser shows visited links in different color). Did I miss something?

[9mido]

Not all websites change the color of visited links though. Google search results seem to do this, but when I visit links on endthefud.org and a few other websites, the visited links do not always change colors when you return back.

Links appearing multiple times on the same page is not good for SEO. As a user who was going through links on endthefud.org, I found myself clicking on the same link again and got frustrated because I already read that article.

[pox]

If visited links don't change colors then it's definitely a CSS problem. I'll get on it.

[9mido]

Ok the CSS is one thing for previously visited links but another CSS thing for coloring duplicate links. If coloring links, have a color key somewhere on the page that tells everybody that this colored link is a duplicate and this colored link is previously visited.

If you want more traffic to endthefud.org from search engine results, it would be best to get rid of duplicate links so that endthefud.org can rank higher on search engines.

I think the whole page should be better organized. Get rid of duplicate links. Show the link only once. But show all categories under that one and only link. Don't show the same link with different anchor tags either.

Maybe try something like this:

Link
Energy, Governments, Etc.

What you could add also is some sort of filtering mechanism for each category. For example, if I filter via a dropdown menu or checkbox by Energy then all links that have the Energy category will be returned. If you filter by Energy and Governments then only links with Energy and Governments combined will be returned.

[pox]

Ok the CSS is one thing for previously visited links but another CSS thing for coloring duplicate links. If coloring links, have a color key somewhere on the page that tells everybody that this colored link is a duplicate and this colored link is previously visited.

Once a link is visited, all its duplicates should appear as visited links in the page.

If you want more traffic to endthefud.org from search engine results, it would be best to get rid of duplicate links so that endthefud.org can rank higher on search engines.

Honestly I don't really care about SEO. The organization of the website is meant to help people easily find all the relevant articles by topic. I don't want people to need to scan a long list of articles of unrelated topics just to tease out articles for a specific one. That's why I think duplicate links are fine. If it hurts SEO so be it. I'm optimizing for this page to be used as a bookmark collection basically where the lookup is by topic, not by article, not by author, etc.

What you could add also is some sort of filtering mechanism for each category. For example, if I filter via a dropdown menu or checkbox by Energy then all links that have the Energy category will be returned. If you filter by Energy and Governments then only links with Energy and Governments combined will be returned.

There are some guys working on a twitter bot like this. Where you can mention the bot and one or more tags and you get relevant articles in a response. It's a nice concept. Doing it on a statically generated website like this isn't trivial, though. Probably requires some custom JS and I'm not sure if that jibes well with Github Pages to be honest. If someone can show me a proof-of-concept page that does this using Github Pages I'll be happy to consider adding tag filtering to endthefud.

[pox]

I've updated the CSS to show visited links in purple. This ameliorates the problem somewhat, although some links (especially those opening in Medium, I've noticed) still don't get colored properly on my mobile device. Maybe it's because they're opened by the Medium app instead of the browser itself... if someone knows how to fix this I'd be eager to learn.

[9mido]

What about putting something like an * or something like (duplicate) next to each duplicate link then to indicate that it is a duplicate? Or just color the duplicate links a different color than unique links?

The only thing I don't like about the CSS purple links is that once you visit endthefud.org again after you clicked a few of the links, those links are still purple and they do not reset themselves. So basically we are back to the original problem again of not knowing which links are duplicate. If you click all of the links on endthefud.org and return, there is no way of knowing which ones are duplicate. I think it would be better to reverse the purple CSS link change you made and label the duplicate links in a different way.