mir/NewPipeExtractor
2
0
Fork 0
mirror of https://github.com/TeamNewPipe/NewPipeExtractor synced 2025-08-30 22:05:18 +00:00
Code Issues Releases Wiki Activity
2,878 Commits 12 Branches 79 Tags
182df6eebe4b3de33d6dea95608dd8bec6f76479
Commit Graph

2165 Commits

Author SHA1 Message Date
litetex
77ee25e3b6 Fix typo 2025-03-05 15:55:46 +01:00
litetex
907fc2ac52 Fix compile problems and optimize TokenStream 2025-03-05 15:45:08 +01:00
litetex
bcf24def8b Cleanup test 2025-03-05 15:44:57 +01:00
litetex
470a719861 Fix compile problems in JavaScript class 2025-03-05 15:44:46 +01:00
litetex
b376539062 Also use rhino engine 2025-03-05 15:44:33 +01:00
litetex
2d3342bf56 Improve tests 2025-03-05 15:44:25 +01:00
dependabot[bot]
7b5d41a741 Bump org.mozilla:rhino from 1.7.15 to 1.8.0 
Bumps [org.mozilla:rhino](https://github.com/mozilla/rhino) from 1.7.15 to 1.8.0.
- [Release notes](https://github.com/mozilla/rhino/releases)
- [Changelog](https://github.com/mozilla/rhino/blob/master/RELEASE-NOTES.md)
- [Commits](https://github.com/mozilla/rhino/commits)

---
updated-dependencies:
- dependency-name: org.mozilla:rhino
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-02-17 21:11:54 +00:00
litetex
c4056f5d5c Fix tests 
Upstream changes are described here: https://github.com/jhy/jsoup/issues/1278

They are effectively irrelevant as YT removes the ``"``/attributes anyway.
 2025-02-17 22:09:44 +01:00
dependabot[bot]
7e21002838 Bump org.jsoup:jsoup from 1.17.2 to 1.18.3 
Bumps [org.jsoup:jsoup](https://github.com/jhy/jsoup) from 1.17.2 to 1.18.3.
- [Release notes](https://github.com/jhy/jsoup/releases)
- [Changelog](https://github.com/jhy/jsoup/blob/master/CHANGES.md)
- [Commits](https://github.com/jhy/jsoup/compare/jsoup-1.17.2...jsoup-1.18.3)

---
updated-dependencies:
- dependency-name: org.jsoup:jsoup
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-02-17 21:06:55 +00:00
Stypox
b1ecb6882f Merge pull request #1277 from litetex/fix-tests 
Fix tests and cleanup other stuff
 2025-02-15 15:23:27 +01:00
litetex
e4081243e7 Improve documentation 
> > I think you can use YoutubeParsingHelper.getTextFromObject()
> That was used there before however I did not choose to use it again.
>
> ``runs`` returns an array with 3 items containing:
> 0: ``<artistname>``
> 1: ``•``
> 2: ``<subscriberCount>``
>
> There is 2 problems with that:
> 1. It's inefficient for extracting the subscription count
> 2. If for some reason the artist now has a name like "2k" the extraction malfunctions

See also: https://github.com/TeamNewPipe/NewPipeExtractor/pull/1277#discussion_r1955148742
 2025-02-15 15:02:30 +01:00
litetex
677c048f1b Only include `jsr305` and not complete spotbugs 
Only ``javax.annotation.Nonull`` or ``javax.annotation.Nullable`` are currently in used which means that spotbugs can and should be removed.

See https://github.com/TeamNewPipe/NewPipeExtractor/issues/1278 for details
 2025-02-15 14:23:33 +01:00
litetex
3ad9f17c53 Revert potentially breaking change 
I expected this class to be located in ``youtube`` as it mostly contains YT specific playlist types, but this assumption was wrong.

See also: d9caa90d23 (r1955161108)
 2025-02-13 21:24:05 +01:00
litetex
d9caa90d23 Fix okhttp deprecation 2025-02-12 22:29:06 +01:00
litetex
4c3918ac4c Removed unused code 2025-02-12 22:09:03 +01:00
litetex
2f2e2df5aa Add recorded mock data 2025-02-12 22:01:17 +01:00
litetex
daac243fc4 Use camelCase 2025-02-12 22:00:54 +01:00
litetex
69d5bf2a89 YouTube 100% Mock coverage 2025-02-12 21:56:02 +01:00
litetex
3b34b82e0f Downloader: Don't force IOException 
Use ``UncheckedIOException`` instead

This no longer forces one to always write ``throws IOException`` in tests
 2025-02-12 21:46:45 +01:00
litetex
177262485a Make all YT tests in MOCK mode use mock data 2025-02-11 22:19:53 +01:00
litetex
cf5df3fcff Improve rate limiting 
* Bandcamp also has rate-limiting (but a lot more req are allowed than with YT)
* Shorten default warm up time
* Fix typo
 2025-02-11 21:50:38 +01:00
litetex
287123b0fd Add rate limiter with default cold factor 2025-02-11 21:49:48 +01:00
litetex
4d7df1486c Fix YT artist getSubscriberCount extraction 2025-02-11 21:35:19 +01:00
litetex
25e4a6f3cf YT: Suggestion test is working again 2025-02-11 21:24:39 +01:00
litetex
ee6bc846ce Re-Enable test as it's no longer broken 2025-02-11 21:17:46 +01:00
litetex
fa99003077 Fix duplicated ; 2025-02-11 21:16:49 +01:00
litetex
1966ad1e52 Use new Bandcamp autocomplete api 2025-02-11 21:16:30 +01:00
litetex
59a2f1aa18 No random errors 2025-02-11 21:10:31 +01:00
litetex
9f94a2912d Remove recording retry/throttle as this is now handled in a more general way 2025-02-11 20:47:14 +01:00
litetex
460e0f77e6 Update test dependencies 2025-02-11 20:46:45 +01:00
litetex
351174a85f Enforce modern TLS1.2+ 
This fixes the following problem:
Bandcamp/their Fastly CDN fails with 403 responses when running tests
* on Windows
* with Java 17+
* and OkHttp's APPROVED_CIPHER_SUITES (default) are used
 2025-02-11 20:43:55 +01:00
litetex
ad5ca4eced Backport/Sync code from NewPipe implementation 2025-02-09 22:06:41 +01:00
litetex
c892cfaafa Fix PeerTube tests 2025-02-09 21:40:05 +01:00
litetex
e1b94c4148 Move into dedicated package 2025-02-09 20:57:21 +01:00
litetex
f1b90985da Remove outdated comment 2025-02-09 20:45:17 +01:00
litetex
1a9df85213 MetaInfo Search is was removed or there is none active anymore 
https://github.com/TeamNewPipe/NewPipeExtractor/issues/1274
 2025-02-09 20:38:38 +01:00
litetex
61ef1699f8 Fix code style 2025-02-09 20:37:59 +01:00
litetex
d746d1b288 Disable irrelevant test 2025-02-09 20:37:50 +01:00
litetex
271a4656da Add missing override annotations 2025-02-09 20:22:59 +01:00
litetex
51586c0957 Disable YT Shorts UI 
https://github.com/TeamNewPipe/NewPipeExtractor/issues/1273
 2025-02-09 20:21:52 +01:00
litetex
451bb1315c YouTube channel mixes no longer exist 
It looks like YT removed this around autumn 2024

Other mentions: https://www.reddit.com/r/youtube/comments/1dhuvyo/channel_mixes_disappeared/
 2025-02-09 20:15:08 +01:00
litetex
ee7d8b042f Channel has a new name 2025-02-09 20:08:49 +01:00
litetex
ddc7f3e33a Fix unexpected error due to malformed url 2025-02-09 20:08:40 +01:00
litetex
6e82a80b26 Disabled age restricted test as it's currently not working 2025-02-09 20:08:29 +01:00
litetex
f0ed76c8c6 Implement rate-limiting 2025-02-09 19:57:04 +01:00
AudricV
96911ae2a4 [YouTube] Fix usage of WEB client headers for all HTML5 URLs in DASH creators 
Also use TVHTML5 user agent for requests from this client in these
DASH manifests creators.
 2025-02-05 09:56:53 +01:00
AudricV
c48d449853 [YouTube] Add ability to get TVHTML5 user agent used 
Also make YoutubeParsingHelper.getOriginReferrerHeaders public, in
order to be used by other extractor classes and improve the name of a
parameter of YoutubeParsingHelper.getVisitorDataFromInnertube.
 2025-02-05 09:56:52 +01:00
AudricV
0952431121 [YouTube] Move InnertubeClientRequestInfo creations in class' methods 
This commits provides methods to get InnertubeClientRequestInfo
instances, which can be used by extractor clients to get visitor data
to pass to PoTokenProvider implementations using YoutubeParsingHelper.

Ability to create custom instances has been removed, but returned
objects can be modified. This is what YoutubeStreamHelper now uses to
set the visitorData property.
 2025-02-05 09:56:52 +01:00
AudricV
4644e1744b [YouTube] Add signatureTimestamp argument to TVHTML5 client requests 
This argument, which has been forgot, is required to get valid
streaming URLs with this client.
 2025-02-05 09:56:52 +01:00
AudricV
61f67854ed [YouTube] Get visitorData for player requests if not provided and do some fixes 
visitorData are get using InnertubeClientRequestInfo and
YoutubeParsingHelper.prepareJsonBuilder, which is replacing the
corresponding method in YoutubeStreamHelper, removed in this commit.

Also fix some bugs like JsonBuilder usages in some places and remove
HLS manifest filtering for the iOS client, as we still use for now
the full player response, in the case having streams requiring poTokens
after some time as a last resort is useful.
 2025-02-05 09:56:52 +01:00