Merge upstream #46
Open
Merge upstream #46
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This adds a new latest_topic_id column to topic_schedule and uses triggers on the topics table to keep it correct. This isn't really ideal, but it will simplify a few things related to scheduled topics by quite a bit. For example, this commit also uses that new data to much more easily populate the list of scheduled topics in a group's sidebar, which previously required a subquery and windowing.
By default, new top-level comments will only be allowed in the latest topic from a particular set of scheduled topics. Replies to existing comments in old topics will still be allowed - this is just intended to prevent the cases where an old scheduled topic gets bumped back up due to a reply and people inadvertently start adding new top-level comments to it instead of the latest one. This should be the correct behavior for most scheduled topics, but it can be disabled for a particular schedule if needed.
I doubt the absence of this would have ever been noticed or that adding this will ever matter for anyone, but I might as well do it properly!
Signed-off-by: Ry Jones <ry@linux.com>
Fixes provisioning of a new VM. Old versions like 2019.2.3 may be moved to an archive and get an HTTP 404 error. Relaxing the pinned version allows setup to find newer patches, such as 2019.2.5. More info: https://github.com/saltstack/salt-enhancement-proposals/blob/752768b1ff900128b192776d950306cba985c99e/accepted/0022-old-releases.md
The generate_site_icons_css cronjob will create this file, but the site won't work before it exists, so there's a (less than 5 min) gap where the site is broken when first set up. This probably won't be noticeable in dev/prod setups, but breaks things like CI setups where everything is getting created freshly each time. This makes sure that the file always exists on initial setup and whenever the Salt states are re-run.
Prevents scrollbar from showing up when there is a subscript on the last line of text. Another option would have been overflow-y: hidden, but that clips the text in the (pathological?) case of deeply nested subscripts.
This message is getting pretty outdated now, and should probably be done in a different way regardless so that it doesn't need to be in the code, especially since forks won't want the same message (or any message). A better approach would probably be a consumer or cronjob watching for new registrations in the event stream.
Trying to change the mode of this file (which often already exists) fails on Windows. It seems fine to just not set it and let it be set to the default.
Tags are stored in the search index as space-separated strings with the periods removed. Searches for "parent.child" tags were failing because of the period. Removing period is okay for now because URL domains are not currently indexed for search.
Use bottom: 100% to make sure the menu does not overlap the button (as with bottom: 0). If it overlaps the button then that interferes with the button click handler.
The "outer" width/height functions also include padding and border. Not including these didn't make a noticeable difference for the left/right flipping (the omissions almost canceled each other out), but the discrepancy is much more noticeable on the top/bottom flipping.
Includes HTML updates to let user click into unfiltered view, when viewing a single tag.
This was done for over 10 months now, there shouldn't really be any old ones left to fix at this point.
This isn't great, but will fix an error that's actively occurring when someone filters to a single tag (tag= query var) and also has a filtered topic tag with a space in it.
I think someone tried to log into the special internal account named "Tildes", which isn't possible (since it has no password), but caused a crash.
Previously, when checking if a link had been posted before, there was no restriction on the time limit, so even posts from years ago would come up. This restricts it to only the last 6 months, which I think is a pretty reasonable time period for reposting.
Salt version 3000 (3000.3 specifically) seems to be working fine now, so it should be safe to stop forcing the old version.
I temporarily pinned two packages that will require more significant updates (webargs in requirements and prospector in requirements-dev). Other than those, everything seemed to upgrade cleanly, except for an issue with mypy that needed a "type: ignore" comment to circumvent. Note that there is currently an issue with Salt's pip module being unable to handle comments in a requirements file that include "-r", so I had to manually edit the two .txt files after using pip-tools to remove all lines with "via -r" comments in them. I've commented about this in an issue on Salt's repo here: saltstack/salt#56514 (comment)
I thought this would be a larger task due to so many of the tools updating to new versions, but the only thing necessary for this upgrade was updating the name of one of the disabled pylint errors.
This was not a fun upgrade. webargs made some major changes to its approaches in 6.0, which are mostly covered here: https://webargs.readthedocs.io/en/latest/upgrading.html To keep using it on Tildes, this commit had to make the following changes: - Write my own wrapper for use_kwargs that changes some of the default behavior. Specifically, we want the location that data is being loaded from to default to "query" (the query string) instead of webargs' default of "json". We also needed to set the "unknown" behavior on every schema to "exclude" so that the schemas would ignore any data fields they didn't need, since the default behavior is to throw an error, which happens almost everywhere because of Intercooler variables and/or multiple use_kwargs calls for different subsets of the data. - All @pre_load hooks in schemas needed to be rewritten so that they weren't modifying data in-place (copy to a new data dict first). Because webargs is now passing all data through all schemas, modifying in-place could result in an earlier schema modifying data that would then be passed in modified form to the later ones. Specifically, this caused an issue with tags on posting a new topic, where we just wanted to treat the tags as a string, but TopicSchema would convert it to a list in @pre_load. - use_kwargs on every endpoint using non-query data needed to be updated to support the new single-location approach, either replacing an existing locations= with location=, or adding location="form", since form data was no longer used by default. - The code that parsed the errors returned by webargs/Marshmallow ValidationErrors needed to update to handle the additional "level" in the dict of errors, where errors are now split out by location and then field, instead of only by field. - A few other minor updates, like always passing a schema object instead of a class, and never passing a callable (mostly just for simplicity in the wrapper).
Adds the HTML Tidy library to the dev version, along with the pytidylib wrapper for it, and a couple of tests that use it to validate the HTML of the home page. Includes a fix to the GitLab "Planned features" link that Tidy considers invalid because it includes some un-encoded characters.
This reverts commit cb7be83. HTML Tidy seems to have various gaps in its validation that we've found already, including one that's pretty much a deal-breaker for Tildes's HTML: it doesn't think that <menu> is a valid parent for <li>. We're looking at alternative validators still.
These were set up to redirect the original locations of the development pages to their new locations inside the instructions folder, but can't be used any more now that we're creating a development folder.
This is simpler than needing to know that --html-validation is the flag to use to make sure that all tests are run, and can stay constant even if we add other types of excluded-by-default tests in the future.
I should have just done this all along, these have been way more trouble than they're worth. If the information is needed, it's always possible to just do a temp run of pip-compile without --no-annotate or use a dedicated tool like pipdeptree.
I think this is going to be a better way to name invoke tasks. The previous naming where a verb was often first made it much harder for anyone to figure out the name of a task that affects a certain thing without always looking through the entire list. For example, if someone is looking for a task that affects the web server, it's much easier to find web-server-reload than reload-web-server. The changes were: - check-code-style -> code-style-check - reload-web-server -> web-server-reload - renew-tls-certificate -> tls-certificate-renew - type-checking -> type-check - update-pip-requirements -> pip-requirements-update
Just rearranges the module so the functions are in alphabetical order (except web_server_reload, which has to be earlier so it can be called as a post-task).
This adds an "Edit title" choice in the actions dropdown for topics on listing pages, instead of needing to go to the comments page. Some pieces of this feel a little hack-ish (like needing to reduce the bottom padding because of the usually-empty div that the title-editing input gets put into), so I'll probably want to try and find a better overall approach to this eventually, but it should do the job for now.
Whoops, only added it to the ones without excerpts, which causes a minor layout issue for the with-excerpt ones.
The __future__ import will be able to be removed as of Python 3.10.
This adds the backend pieces (no interface yet) to configure Lua scripts
that will be applied to topics and comments due to different events.
Initially, it only supports running a script when a new topic or comment
is posted. For example, here is a Lua script that would prepend a new
topic's title with "[Text] " or "[Link] " depending on its type, as well
as replace its tags with either "text" or "link":
function on_topic_post (topic)
if (topic.is_text_type) then
topic.title = "[Text] " .. topic.title
topic.tags = {"text"}
elseif (topic.is_link_type) then
topic.title = "[Link] " .. topic.title
topic.tags = {"link"}
end
end
There can be a global script as well as group-specific scripts, and the
scripts are sandboxed, with limited access to data as well as being
restricted to a subset of Lua's built-in functions. The Lua sandboxing
code comes from Splash (https://github.com/scrapinghub/splash). It will
need to be modified, but this commit keeps it unmodified so that future
changes can be more easily tracked by comparing to the original state of
the file.
The sandboxing also includes some restrictions on number of instructions
and memory usage, but this might be more effectively managed on the OS
level. More research will still need to be done on security and resource
restrictions before this feature can be safely opened to users.
Previously, rate limits had to apply to a particular user or a particular IP address, or both. This adds support for global rate-limits, where the limit will apply to everyone trying to perform the action. This probably won't be used much overall, but might be necessary for certain cases where something abusive is happening and it can't be easily blocked by user or IP. This is a bit ugly and would probably be better implemented by having a separate class that inherits from RateLimitedAction or something similar, but it will do the job.
People are still continuing to try to abuse the donate page to check stolen credit card numbers, and last night there was a massive burst of attempts coming from many IPs, so the current rate-limiting wasn't able to block most of it. Luckily Stripe blocked all of the charges this time, but I can't keep risking another incident where Tildes is the source of a bunch of fraudulent charges. This adds a global rate-limit to the donate page that should never get hit during normal usage. Hopefully this will be enough to keep the abuse away from the page when it stops working for them relatively quickly.
If user types "tag1 tag2" then adds a comma between, it should respect the comma to give "tag1" and "tag2". We use keydown and setTimeout because keyup works on a keyboard but not reliably on mobile. Nonzero timeout is needed or else the comma is sometimes inserted too late and not seen by addChip(), tested on desktop Firefox.
Some emoji variants require a zero-width joiner, and they were being broken by the current code that stripped them out.
There was the potential for an IndexError here, with a string that started with a zero-width joiner and had at least one more character afterwards.
Contributor
Author
|
Looks like this is the CI error: I'll have to look into it later. |
…to a group. When logged in the group has a validation error that is not worth it to fix right now.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request is probably not ready for prime time, as we may have lost some of the visual improvements in #11
Need to check. But it runs, at least.