| Age | Commit message (Collapse) | Author |
|---|
|
I just realised that `commitNewEntries()` was not sorting articles properly before insertion in database when using MySQL: Articles were not sorted by publication date as expected from the temporary table before insertion in the final table. MySQL was not picking the correct field, so fixed with an explicit alias.
Discovered because I did some tests with MySQL in https://github.com/FreshRSS/FreshRSS/pull/6957
At the same time, I did the same change for PostgreSQL and SQLite although those were not affected.
|
|
Allow the use of regex `\b` for word boundary (and `\B` for the opposite) even when using PostgreSQL.
Follow up of:
* https://github.com/FreshRSS/FreshRSS/pull/6706
For instance, `intitle:/\bnew\B/` will find *newest* but not *new* nor *renewal*.
Useful in particular to minimise the differences between PHP and database in:
* https://github.com/FreshRSS/FreshRSS/pull/7959
|
|
* Fix SQLite GREATEST()
fix https://github.com/FreshRSS/FreshRSS/pull/8105
Related to https://github.com/FreshRSS/FreshRSS/pull/7886
|
|
* feat: Add user modified functionality
Closes https://github.com/FreshRSS/FreshRSS/issues/7862
Changes proposed in this pull request:
This is an implementation of the proposed feature. It allows entries to have a new field that will be updated whenever an item is marked as read/unread or bookmark/removed from bookmarks. And a new sort criteria to sort by it.
How to test the feature manually:
1. Mark items from a feed as read/unread
2. Mark items from a feed as bookmark / remove bookmark
3. Sort by the new criteria
* feat: Add sort functionality
* feat: Add sort nav button
* fix: Use correct migrations
* fix: Add internationalization
* fix: Linter errors
* chore: PR comments
* Update app/i18n/fr/index.php
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
* Update app/i18n/pl/index.php
Co-authored-by: Inverle <inverle@proton.me>
* Update app/i18n/nl/index.php
Co-authored-by: Frans de Jonge <fransdejonge@gmail.com>
* make fix-all
* Fixes
* More fixes sort
* Fix wrong index
* Fix unneeded column
* Fix auto-create indexes
* Some copilot suggestions
* One more fix
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
---------
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
Co-authored-by: Inverle <inverle@proton.me>
Co-authored-by: Frans de Jonge <fransdejonge@gmail.com>
|
|
* Recovery: skip broken entries during CLI export/import
fix https://github.com/FreshRSS/FreshRSS/discussions/7927
```
25605/25605 (48 broken)
```
Help with *database malformed* or other corruption.
* Compatibility multiple databases
|
|
Closes #3789, #6503
Icon setting when no custom icon is set yet:
- `Change...` button opens a file dialog, and after selecting a file shows the chosen icon in the preview on the left. `Submit` must be clicked after selecting the icon.
- `Reset to default` changes the preview icon to the default one, and also requires `Submit` to be clicked to apply the changes.
Full list of changes:
- CSP now includes `blob:` in `img-src` for
- `indexAction()` and `feedAction()` in `subscriptionController.php`
- all of the view actions in `indexController.php`
- Introduce new attribute `customFavicon (boolean)` for feeds that indicates if the feed has a custom favicon
- `hashFavicon()` in `Feed.php` is dependent on this attribute
- `hashFavicon()` has a new parameter called `skipCache (boolean)` that allows the reset of the favicon hash for the Feed object
- `resetFaviconHash()` just calls `hashFavicon(skipCache: true)`
- `f.php` URLs now have the format of `/f.php?h=XXXXX&t=cachebuster`, where the `t` parameter is only used for serving custom favicons
- if `t` parameter is set, `f.php` returns a `Cache-Control: immutable` header
- `stripos` and `strpos` were changed to `str_contains` in various places (refactor)
- JS for handling the custom favicon configuration logic is in `extra.js` inside `init_update_feed()` which is called when feed configuration is opened from the aside or when the subscription management page with the feed is loaded
- Server-side code for uploading the icon in `subscriptionController.php` under `feedAction()`
- Errors that may occur during the setting of a custom favicon:
- Unsupported image file type (handled only server-side with `isImgMime()`)
- When the file is bigger than 1 MiB (default), handled both client-side and server-side
- Standard feed error when `updateFeed()` fails
- JS vars `javascript_vars.phtml` are no longer escaped with `htmlspecialchars()`, instead with json encoding,
- CSS for disabled buttons was added
- Max favicon file size is configurable with the `max_favicon_upload_size` option in `config.php` (not exposed via UI)
- Custom favicons are currently deleted only when they are either reset to the default icon, or the feed gets deleted. They do not get deleted when the user deletes their account without removing their feeds first.
- ` faviconPrepare()` and `faviconRebuild()` are not allowed to be called when the `customFavicon` attribute is `true`
- New i18n strings:
- `'sub.feed.icon' => 'Icon'`
- `'sub.feed.change_favicon' => 'Change…'`
- `'sub.feed.reset_favicon' => 'Reset to default'`
- `'sub.feed.favicon_changed_by_ext' => 'The icon has been set by the <b>%s</b> extension.'`
- `'feedback.sub.feed.favicon.too_large' => 'Uploaded icon is too large. The maximum file size is <em>%s</em>.'`
- `'feedback.sub.feed.favicon.unsupported_format' => 'Unsupported image file format!'`
- Extension hook `custom_favicon_hash`
- `setCustomFavicon()` method
- `resetCustomFavicon()` method
- `customFaviconExt` and `customFaviconDisallowDel` attributes
- example of usage: https://github.com/FreshRSS/Extensions/pull/337
- Extension hook `custom_favicon_btn_url`
- Allows extensions to implement a button for setting a custom favicon for individual feeds by providing an URL. The URL will be sent a POST request with the `extAction` field set to either `query_icon_info` or `update_icon`, along with an `id` field which describes the feed's ID.
|
|
Add option to sort results by received date (existing, default), publication date, title, URL (link), random.
fix https://github.com/FreshRSS/FreshRSS/issues/1771
fix https://github.com/FreshRSS/FreshRSS/issues/2083
fix https://github.com/FreshRSS/FreshRSS/issues/2119
fix https://github.com/FreshRSS/FreshRSS/issues/2596
fix https://github.com/FreshRSS/FreshRSS/issues/3204
fix https://github.com/FreshRSS/FreshRSS/issues/4405
fix https://github.com/FreshRSS/FreshRSS/issues/5529
fix https://github.com/FreshRSS/FreshRSS/issues/5864
fix https://github.com/FreshRSS/Extensions/issues/161
URL parameters:
* `&sort=id` (current behaviour, sorting according to newest received articles)
* `&sort=date` (publication date, which is not indicative of how new an article is)
* `&sort=title`
* `&sort=link`
* `&sort=rand` (random order - which disables infinite scrolling, at least for now)
combined with `&order=ASC` or `&order=DESC`
## Implementation notes
The sorting criteria by *received date* (id), which is the default, and which was the only one before this PR, is the one that has the best sorting characteristics:
* *uniqueness*: no entries have the exact same received date
* *monotonicity*: new entries always have a higher received date
* *performance*: this field is efficiently indexed in database for fast usage, including for paging (indexing could also be done to other fields, but with lower effective performance)
In contrary, sorting criteria such as by *publication date*, by *title*, or by *link* are neither unique nor monotonic. In particular, multiple articles may share the same *publication date*, and we may receive articles with a *publication date* far in the future, and then later some new articles with a *publication date* far in the past.
To understand why sorting by *publication date* is problematic, it helps to think about sorting by *title* or by *link*, as sorting by *title* and by *publication date* share more or less the same characteristics.
### Problem 1: new articles
New articles may be received in the background after what is shown on screen, and before the next user action such as *mark all as read*. Due to the lack of *monotonicity* when sorting by e.g. *publication date* or *title*, users risk marking as read a batch of articles containing some fresh articles without seeing them.
Mitigation: A parameter `idMax` tracks the maximum ID related to a batch of actions such as *mark all as read* to exclude articles received after those that are displayed.
### Problem 2: paging / pagination
When navigating articles, only a few articles are displayed, and a new "page" of articles needs to be received from the database when scrolling down or when clicking the button to show more articles. When sorting by e.g. *publication date* or *title*, it is not trivial to show the next page without re-showing some of the same articles, and without skipping any. Indeed, views are often with additional criteria such as showing only unread articles, and users may mark some articles as read while viewing them, hereby removing some articles from the previous pages. And like for *Problem 1*, new articles may have been received in the background. Consequently, it is not possible to use `OFFSET` to implement pagination (so the patches suggested by a few users were wrong due to that, in particular).
Mitigation: `idMax` is also used (just like for *Problem 1*) and a *Keyset Pagination* approach is used, combining an unstable sorting criterion such as *publication date* or *title*, together with *id* to ensure stable sorting. (So, 2 sorting criteria + 1 filter criteria)
See e.g. https://www.alwaysdeveloping.net/dailydrop/2022/07/01-keyset-pagination/
### Problem 3: performance
Sorting by anything else than *received date* (id) is doomed to be slow(er) due to the combination of 3 criteria (see *Problem 2*). An `OFFSET` approach (which is not possible anyway as explained) would be even slower. Furthermore, we have no SQL index at the moment, but they would not necessarily help much due to the multiple sorting criteria needed and involving some `OR` logic which is difficult to optimise for databases.
The nicest syntax would be using tuples and corresponding indexes, but that is poorly supported by MySQL https://bugs.mysql.com/bug.php?id=104128
Mitigation: a compatibility SQL syntax is used to implement *Keyset Pagination*
### Problem 4: user confusion
Several users have shown that they do not fully understand the difference between *received date* and *publication date*, and particularly not the pitfalls of *publication date*.
Mitigation: the menus to mark-as-read *before 1 day* and *before 1 week* are disabled when sorting by anything else than *received date*. Likewise, the separation headers *Today* and *Yesterday* and *Before yesterday* are only shown when sorting by *received date*.
Again here, to better understand why, it helps to think about sorting by *title* or by *link*, as sorting by *title* and by *publication date* share more or less the same characteristics.
* [ ] We should write a Q&A and/or documentation about the problems associated to *sorting by publication date*: risks of not noticing new publication, of inadvertently marking them as read, of having some articles with a date in the future hanging at the top of the views (vice versa when sorting in ascending order), performance, etc.
### Problem 5: APIs
Sorting by anything else than *received date* breaks the guarantees needed for a successful synchronisation via API.
Mitigation: sorting by *received date* is ensured for all API calls.
|
|
* PHPStan 2.0
fix https://github.com/FreshRSS/FreshRSS/issues/6989
https://github.com/phpstan/phpstan/releases/tag/2.0.0
https://github.com/phpstan/phpstan/blob/2.0.x/UPGRADING.md
* More
* More
* Done
* fix i18n CLI
* Restore a PHPStan Next test
For work towards PHPStan Level 10
* 4 more on Level 10
* fix getTagsForEntry
* API at Level 10
* More Level 10
* Finish Minz at Level 10
* Finish CLI at Level 10
* Finish Controllers at Level 10
* More Level 10
* More
* Pass bleedingEdge
* Clean PHPStan options and add TODOs
* Level 10 for main config
* More
* Consitency array vs. list
* Sanitize themes get_infos
* Simplify TagDAO->getTagsForEntries()
* Finish reportAnyTypeWideningInVarTag
* Prepare checkBenevolentUnionTypes and checkImplicitMixed
* Fixes
* Refix
* Another fix
* Casing of __METHOD__ constant
|
|
* Regex search
fix https://github.com/FreshRSS/FreshRSS/issues/3549
* Fix PHPStan
* Fix escape
* Fix ungreedy
* Initial support for regex search in PostgreSQL and MySQL
* Improvements, support MySQL
* Fix multiline
* Add support for SQLite
* A few tests
* Added author: and inurl: support, documentation
* author example
* Remove \b for now
* Disable regex sanitization for now
* Fix getInurlRegex
* getNotInurlRegex
* Quotes for inurl:
* Fix test
* Fix quoted tags + regex for tags
https://github.com/FreshRSS/FreshRSS/issues/6761
* Fix wrong regex detection
* Add MariaDB
* Fix logic
* Increase requirements for MySQL and MariaDB
Check support for multiline mode in MySQL
* Remove sanitizeRegexes()
* Allow searching HTML code
Allow searching for instance `/<pre>/`
Fix https://github.com/FreshRSS/FreshRSS/issues/6775#issuecomment-2331769883
* Doc regex search HTML
* Fix Doctype
|
|
* PHP 8.3 #[\Override]
https://php.watch/versions/8.3/override-attr
With PHPStan `checkMissingOverrideMethodAttribute` https://phpstan.org/config-reference#checkmissingoverridemethodattribute
And modified the call to phpstan-next on the model of https://github.com/FreshRSS/Extensions/pull/228 (more robust than the find method, which gave some strange errors)
* Update extension example accordingly
|
|
* Little's optimisations and booleans in conditions
* Apply strict type
* Apply strict type
* Apply strict type
* Fix multiple bugs with PHP 8.2 and 8.3
* Many declares missing, more errors fixed
* Apply strict type
* Another approach
* Stronger typing for Minz_Session
* Fix case of SQLite
---------
Co-authored-by: Luc <sanchezluc+freshrss@gmail.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
|
|
https://php.net/pdo.errorinfo has slightly changed signature
|
|
Should help with some DB lock issues.
Complete https://github.com/FreshRSS/FreshRSS/pull/3558 after https://github.com/FreshRSS/FreshRSS/pull/5625 already cherry-picked from it.
* Removed auto-update of MySQL GUID case sensitivity
https://github.com/FreshRSS/FreshRSS/pull/2078
* Contributed to a DB lock in https://github.com/FreshRSS/FreshRSS/issues/5008
Also removed the following non-problematic auto-updates, simply because they were older than the above ones
* Auto-create custom labels (1.12.0) https://github.com/FreshRSS/FreshRSS/pull/2027
* Auto-add JSON column for feeds (1.11.0) https://github.com/FreshRSS/FreshRSS/pull/1838
* Auto-create temporary tables (1.7.0) https://github.com/FreshRSS/FreshRSS/pull/1470
|
|
* Remove FreshRSS_Searchable for better types
The interface was not used, and it was preventing more precise types for the different `searchById()` methods, as they each have different input and output types.
* Consistent entry ID
Entry IDs (which are 64-bit integers) must be processed as string to be compatible with 32-bit platforms
* Fix type
* A few more related types
* PHPStan level 6
* Some more casts needed
* String cast for htmlspecialchars
|
|
* Add database field attributes for entries
Just like we already have for categories, feeds, etc.
No core use yet, but allows in particular extensions to save per-entry data
* Fix PHPStand
* Fix wrong variable
|
|
* New possibility to invoke user queries from a search expression
From the search field: `S:"My query"`.
Can be combined with other filters such as `S:"My query" date:P3d` as long as the user queries do not contain `OR`.
A use-case is to have an RSS filter with a stable address or an external API call with the ability to update the user query.
* Draft of parenthesis logic
* More draft
* Working parenthesis (a OR b) (c OR d)
* Working (A) OR (B)
* Support nested parentheses + unit tests + documentation
* search:MySearch and S:3
|
|
* More PHP type hints for Fever
Follow-up of https://github.com/FreshRSS/FreshRSS/pull/4201
Related to https://github.com/FreshRSS/FreshRSS/issues/4200
* Detail
* Draft
* Progress
* More draft
* Fix thumbnail PHP type hint
https://github.com/FreshRSS/FreshRSS/issues/4215
* More types
* A bit more
* Refactor FreshRSS_Entry::fromArray
* Progress
* Starts to work
* Categories
* Fonctional
* Layout update
* Fix relative URLs
* Cache system
* Forgotten files
* Remove a debug line
* Automatic form validation of XPath expressions
* data-leave-validation
* Fix reload action
* Simpler examples
* Fix column type for PostgreSQL
* Enforce HTTP encoding
* Readme
* Fix get full content
* target="_blank"
* gitignore
* htmlspecialchars_utf8
* Implement HTML <base>
And fix/revert `xml:base` support in SimplePie https://github.com/simplepie/simplepie/commit/e49c578817aa504d8d05cd7f33857aeda9d41908
* SimplePie upstream PR merged
https://github.com/simplepie/simplepie/pull/723
|
|
* Fix Fever 32 bit ID issue + more PHP type hints
#fix https://github.com/FreshRSS/FreshRSS/issues/4200
Follow up and fix regression from https://github.com/FreshRSS/FreshRSS/pull/4110
* More PHP type hints with PHPStan
* Fix pull problem
* Avoid more nulls
|
|
#fix https://github.com/FreshRSS/FreshRSS/issues/3402
Explicit `INSERT OR IGNORE` / `ON CONFLICT DO NOTHING` for the
`addEntry()` method, which does expect some duplicates.
|
|
* PDO refactor
* Automatic prefix when using the syntax `_tableName`
* Uniformity: MySQL is now PDO::ATTR_EMULATE_PREPARES = false just like SQLite and PostgreSQL, with consequences such as only one statement per query
* Use PDO methods exec(), query(), prepare() + execute() in a more efficient way
* Remove auto-update SQL code for versions older than FreshRSS 1.5 (3 years old)
* The name of the default category is set in PHP instead of in the DB (simplies SQL and allows changing the name according to the FreshRSS language)
* Rename `->bd` to `->pdo` (less of a frenshism, and more informative)
* Fix some requests, which were not compatible with MySQL prepared statements
* Whitespace
* Fix syntax for PostgreSQL sequences
+ MySQL install
* Minor formatting
* Fix lastInsertId for PostgreSQL
* Use PHP 5.6+ const
Take advantage of https://github.com/FreshRSS/FreshRSS/pull/2527
https://www.php.net/manual/en/migration56.new-features.php
* A bit of forgotten PHP 5.6 simplification for cURL
* Forgotten $s
* Mini fix custom user config
https://github.com/FreshRSS/FreshRSS/pull/2490/files#r326290346
* More work on install.php but not finished
* install.php working
* More cleaning of PDO in install
* Even more simplification
Take advantage of PDO->exec() to run multiple statements
* Disallow changing the name of the default category
https://github.com/FreshRSS/FreshRSS/pull/2522#discussion_r326967724
|
|
* CLI to export/import any database to/from SQLite
Require PHP 5.5+ https://github.com/FreshRSS/FreshRSS/pull/2495
* Travis
* Execution rights
* Fix wrong static fields
* Fix MySQL bad default buffering
https://stackoverflow.com/questions/6895098/pdo-mysql-memory-consumption-with-large-result-set/6935271#6935271
https://php.net/manual/ref.pdo-mysql
* Fix count on progression
* Avoid static DB information
To ease working with two DBs at the same time
* Less static, simplify
Needs some testing
* Small corrections
* Special case for SQLite to SQLite
* Modify special case for SQLite
* Remove special case for SQLite
More uniform logic for the 3 databases.
Fix wrong DROP TABLE for SQLite.
* Drop indexes
* Revert "Drop indexes"
This reverts commit f28d2bae0935745c1c74ea38f2ee083f3fd4bf9d.
* Fix deletion
* Fix classic export
* Update cli/README.md
Co-Authored-By: Marien Fressinaud <dev@marienfressinaud.fr>
* Addressing part of review
* Remove goto :cry:
* Travis
* Comment for SQLite case
* Fix missing fields when inserting
|
|
* No old ID
https://github.com/FreshRSS/FreshRSS/issues/2273
* PostgreSQL insert or ignore
|
|
* First draft of custom tags
https://github.com/FreshRSS/FreshRSS/issues/928
https://github.com/FreshRSS/FreshRSS/issues/1367
* SMALLINT to BIGINT for id_entry
And uppercase SQL types
* Fix layout for unreads
* Start UI menu
* Change menu order
* Clean database helpers
https://github.com/FreshRSS/FreshRSS/pull/2027#discussion_r217971535
* Travis rules do not understand PostgreSQL constants
Grrr
* Tag controller + UI
* Add column attributes to tags
* Use only favicon for now, for label
* Fix styling for different themes
* Constant for maximum InnoDB index length in Unicode
https://github.com/FreshRSS/FreshRSS/pull/2027#discussion_r219052200
(I would have personnally prefered keeping the readability of a real
value instead of a constant, in this case of many SQL fields)
* Use FreshRSS_Factory::createCategoryDao
* Add view of all articles containing any tag
* Fix search in tags
* Mark as read tags
* Partial auto-update unread tags
* More auto update tag unreads
* Add tag deletion
* Do not purge tagged articles
* Minor comment
* Fix SQLite and UI bug
* Google Reader API support for user tags
Add SQL check that tag names must be distinct from category names
* whitespace
* Add missing API for EasyRSS
* Compatibility SQLite
Problematic parentheses
* Add SQL DISTINCT for cases with multiple tags
* Fix for PostgreSQL
PostgreSQL needs some additional type hint to avoid "could not determine
data type of parameter $1"
http://www.postgresql-archive.org/Could-not-determine-data-type-of-parameter-1-tp2171092p2171094.html
|
|
CLI optimize database https://github.com/FreshRSS/FreshRSS/issues/1583
And VACUUM in SQLite https://github.com/FreshRSS/FreshRSS/issues/918
Add VACUUM for PostgreSQL (Not tested yet)
|
|
* [ci] Add Travis
* Exclude some libs
* Semi-auto whitespace fixes
* line length in SQLite
* Exclude tests from line length
* Feed.php line length
* Feed.php: get rid of unnecessary concat
* Feed.php: line length
* bootstrap.php: no newline at end of file
* Allow concatenating across multiple lines
* Add Travis badge
* do-install line length
* update-or-create-user line length
* cli/create-user line length
* tests/app/Models/SearchTest.php fix indentation
* tests/app/Models/UserQueryTest.php fix indentation
* tests/app/Models/CategoryTest.php fix indentation
* [fix] PHP 5.3 on precise
* cli/do-install no spaces
* cli/list-users line length
* cli/reconfigure line length
* empty catch statements
* api/index line length nonsense
* spaces before semicolon
* app/Models/EntryDAO bunch of indentation
* extra blank lines
* spaces before comma in function call
* testing tabwidth
* increase to 10
* comment out tabwidth line
* try older phpcs version 3.0.0RC4
* line length exception for app/install.php
* proper spaces
* stray spaces in i18n
* Minz/ModelPdo line length
* Minz whitespace
* greader line length
* greader elseif placement
* app/Models/Feed.php spacing in function argument
* ignore php 5.3
* app/Models/ConfigurationSetter.php stray whitespace
* EntryDAOSQLite line length
* I vote for higher max line length =P
* ignore SQL
* remove classname complaint
* line length/more legible SQL
* ignore line length nonsense
* greader line length
* feedController issues
* uppercase TRUE, FALSE, NULL
* revert
* importExportController lowercase null
* Share.php default value not necessary because ! is_array () a few lines down
* CategoryDAO constants should be UPPERCASE
* EntryDAO reduce line length
* contentious autofix
* Allow failures on all versions of PHP except 7.1 because reasons
|
|
|
|
to entry
Fixes #1610 by first ignoring all entries from entrytmp which exists
already inside the entry table.
|
|
https://github.com/FreshRSS/FreshRSS/issues/530
|
|
|
|
https://github.com/FreshRSS/FreshRSS/issues/416
Based on @Damstre work https://github.com/FreshRSS/FreshRSS/pull/1071
Not tested
|
Alexandre Alapetite
Federico Scodelaro
Inverle
Luc SANCHEZ
Frans de Jonge
Adrien Dorsaz
