mir/NewPipeExtractor
2
0
Fork 0
mirror of https://github.com/TeamNewPipe/NewPipeExtractor synced 2025-08-31 06:15:37 +00:00
Code Issues Releases Wiki Activity
2,907 Commits 12 Branches 79 Tags
260ba4749a60d620fda400db069fed2e5727d35a
Commit Graph

2183 Commits

Author SHA1 Message Date
Stypox
80e42aa584 [YouTube] Fix search filters messing with which lockup formats are extracted 2025-07-28 15:56:10 +02:00
TobiGr
0378fdee5c Fix checkstyle 2025-07-27 12:34:30 +02:00
TobiGr
e9dbcba308 Update USER_AGENT to Firefox ESR 140 2025-07-27 09:41:09 +02:00
FineFindus
352fae640f [YouTube] Correctly set uploaderUrl for lockup content type items 
Fixes an issue, where the uploader URL for related items would be incorrect,
due to the `YoutubeChannelLinkHandlerFactory.getUrl` expecting the id with a
`channel/` prefix. However, the `browseId` used to extract the channel id is
missing this prefix.

Ref: https://github.com/TeamNewPipe/NewPipeExtractor/pull/1320

Fix getUploaderUrl not resolving correct url
 2025-07-26 15:24:11 +02:00
litetex
7adbc48a0a Merge pull request #1332 from litetex/fix-all-tests 
Fix all tests and make everything work offline/with mocks.
Remove old mocks and generate new ones with new structure.

Remove SoundCloud "Top 50" kiosk.
 2025-07-18 12:27:50 +02:00
dependabot[bot]
3e6f1fa018 Bump protobufVersion from 4.30.2 to 4.31.1 
Bumps `protobufVersion` from 4.30.2 to 4.31.1.

Updates `com.google.protobuf:protobuf-javalite` from 4.30.2 to 4.31.1

Updates `com.google.protobuf:protoc` from 4.30.2 to 4.31.1
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/protobuf_release.bzl)
- [Commits](https://github.com/protocolbuffers/protobuf/compare/v4.30.2...v4.31.1)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-javalite
  dependency-version: 4.31.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: com.google.protobuf:protoc
  dependency-version: 4.31.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-07-17 09:51:21 +00:00
Stypox
35cc5a77b5 [YouTube] Support LOCKUP_CONTENT_TYPE_PODCAST 
Also extend support for LOCKUP_CONTENT_TYPE_VIDEO
 2025-07-16 14:48:06 +02:00
AudricV
3ba3dbc4d1 [YouTube] Update channel mocks using playlists 2025-07-16 13:26:39 +02:00
AudricV
dbca2b4e1f [YouTube] Update playlists mocks 2025-07-16 13:26:39 +02:00
AudricV
88efec6e4d [YouTube] Enable YoutubePlaylistExtractorTest.ShortsUI.testMoreRelatedItems 
As continuations are now again available for playlists with Shorts UI, more
items' tests can be enabled.
 2025-07-16 13:26:39 +02:00
AudricV
9415e1bbef [YouTube] Workaround playlists' Shorts UI 
YouTube doesn't return currently a continuation, if applicable, for
Shorts UI playlists, restricting access to the 100th first items. The
reel items returned don't give also upload date, uploader info and
precise view count.

Using a continuation which requests the first page of the playlist
allows currently to get access to continuations, if applicable, and
also standard video elements instead of Shorts ones, making extraction
of upload date, uploader info and precise view count again possible.

This method is used for all playlist types, the original request is
still made, but now only returns what we need.

It requires to add a protocol buffer definition file, for which its
structure is based on reverse engineering of playlists continuations
sent by WEB InnerTube client, received from InnerTube responses.

Java classes of this file are generated for the Java Lite runtime of
Protobuf with the Protobuf Gradle plugin, as the lite version is enough
for our use cases. This plugin ships in JARs Protobuf definitions,
which should be avoided.

As Protobuf classes are parsed by Checkstyle checks and do not follow
our style rules at all, an exclusion rule has been for them.
 2025-07-16 13:26:38 +02:00
AudricV
e12094f483 [YouTube] Add utility method to make InnerTube requests with custom 
query parameters

This can be used to pass Google APIs standard parameters to requests.
 2025-07-16 13:26:38 +02:00
Stypox
4d1ea24c5c Improve comments 2025-07-16 13:22:13 +02:00
litetex
b6b395061c Small live stream improvements and cleanup 
Thanks to Stypox review

* Improved live stream related code
* Fixed Javadoc

Co-Authored-By: Stypox <stypox@pm.me>
 2025-07-15 22:26:01 +02:00
litetex
64e4bafee1 Remove not needed comment 2025-07-15 22:06:02 +02:00
litetex
8e07a76c71 Do not parse duration when live stream 
Also cache the streamtype
 2025-07-14 20:39:12 +02:00
litetex
b15c0dccc5 Cleanup 
Co-Authored-By: Audric V. <74829229+AudricV@users.noreply.github.com>
 2025-07-14 20:31:18 +02:00
litetex
4d7a494757 Update to latest nanojson version of TeamNewPipe 2025-07-13 20:59:21 +02:00
litetex
182df6eebe Handle situations where there are multiple uploaders 2025-07-13 20:59:21 +02:00
litetex
ef6a53ebb3 Prevent NPE on live streams and cache correctly 2025-07-13 20:59:21 +02:00
litetex
ba2efde599 Updated test mocks to also check for `YoutubeStreamInfoItemLockupExtractor` 2025-07-13 20:59:20 +02:00
litetex
d9ba25ce85 Update docs, add caching and removed unused code 2025-07-13 20:59:20 +02:00
litetex
708587a992 Cleanup 2025-07-13 20:59:20 +02:00
litetex
f15190f8f9 Change back to private 2025-07-13 20:59:20 +02:00
litetex
e08747f339 Add support for detecting livestreams 2025-07-13 20:59:20 +02:00
litetex
af4b1b95d8 Added fallback for videoId 2025-07-13 20:59:20 +02:00
litetex
9fb60c7ef5 Use already existing method 2025-07-13 20:59:20 +02:00
litetex
939d5e1f74 There is no formatting 2025-07-13 20:59:20 +02:00
litetex
a18e98adbb Remove copyright from original file 2025-07-13 20:59:20 +02:00
litetex
b07b3dae7c Basic implementation of `YoutubeStreamInfoItemLockupExtractor` 2025-07-13 20:59:20 +02:00
litetex
ed37a429d1 Use my nanojson version which has some shortcut methods 2025-07-13 20:59:20 +02:00
Tobi
71b335128e Merge pull request #1308 from watermelon42/3783_Import_Soundcloud_likes 
[SoundCloud] Add support for likes channel tab
 2025-07-13 03:16:34 -07:00
Tobi
c0ff21af93 Merge pull request #1330 from litetex/speed-up-n-parameter-extraction 
[YouTube] Speed up n parameter presence check
 2025-07-12 05:10:49 -07:00
litetex
249f8f83e7 Updated mocks 2025-07-11 20:19:12 +02:00
litetex
0dfaeb20fa Added doc for related code 2025-07-11 20:12:50 +02:00
FineFindus
1915dc6904 [YouTube] Allow both types of string quotes 2025-07-11 08:27:44 +02:00
FineFindus
a847fdb330 [YouTube] Allow lowercase global array names 2025-07-11 08:27:44 +02:00
FineFindus
d914e86a82 [YouTube] extract global variable for signature deobfuscation 
Extracts the global variable used in newer signature deobfuscation functions,
such as the one found in the `59b252b9` player.
 2025-07-11 08:27:44 +02:00
FineFindus
235dd7cc55 [YouTube] add new signature regex 
Adds a new regex to find the signature deobfuscation function, which
works for the new `20830619` player.

Closes: https://github.com/TeamNewPipe/NewPipeExtractor/issues/1287
Ref: https://github.com/TeamNewPipe/NewPipeExtractor/issues/1287#issuecomment-2764573093
 2025-07-11 08:27:44 +02:00
litetex
1b8864afb8 Merge pull request #1302 from FineFindus/fix/throttling-function-regex 
[YouTube] add new deobfuscation function name regex
 2025-07-10 23:48:08 +02:00
litetex
12f7615e4a Updated mocks for new deobfuscation function name regex 2025-07-10 23:46:28 +02:00
litetex
5184231479 Do a quick check if the n parameter is even present before exec regex 2025-07-10 23:37:54 +02:00
AudricV
26bcfcdeaf [YouTube] Detect members first and members only shorts 
This fixes view count extraction error for such content, as the view
count is replaced by a text describing the availability of the short.

Also made YoutubeShortsLockupInfoItemExtractor package private, as it
doesn't need to be public.
 2025-07-09 18:05:06 +02:00
watermelon42
5e1a1d51b6 Handle liked playlists using the SoundcloudPlaylistInfoItemExtractor 2025-07-08 18:44:51 +03:00
Tobi
22f7cd4a6f Merge pull request #1309 from TeamNewPipe/dependabot/gradle/org.jsoup-jsoup-1.21.1 
Bump org.jsoup:jsoup from 1.19.1 to 1.21.1
 2025-07-08 05:08:08 -07:00
dependabot[bot]
f06b01f036 Bump org.jsoup:jsoup from 1.19.1 to 1.21.1 
Bumps [org.jsoup:jsoup](https://github.com/jhy/jsoup) from 1.19.1 to 1.21.1.
- [Release notes](https://github.com/jhy/jsoup/releases)
- [Changelog](https://github.com/jhy/jsoup/blob/master/CHANGES.md)
- [Commits](https://github.com/jhy/jsoup/compare/jsoup-1.19.1...jsoup-1.21.1)

---
updated-dependencies:
- dependency-name: org.jsoup:jsoup
  dependency-version: 1.21.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-07-07 21:00:47 +00:00
dependabot[bot]
8f28ab3fea Bump com.google.code.gson:gson from 2.12.1 to 2.13.1 
Bumps [com.google.code.gson:gson](https://github.com/google/gson) from 2.12.1 to 2.13.1.
- [Release notes](https://github.com/google/gson/releases)
- [Changelog](https://github.com/google/gson/blob/main/CHANGELOG.md)
- [Commits](https://github.com/google/gson/compare/gson-parent-2.12.1...gson-parent-2.13.1)

---
updated-dependencies:
- dependency-name: com.google.code.gson:gson
  dependency-version: 2.13.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-07-07 21:00:04 +00:00
Tobi
83b12ba93a Merge pull request #1317 from AudricV/soundcloud-drm-free-aac-160k 
[Soundcloud] Add support for AAC 160k unprotected stream and update Creative Commons stream test
 2025-07-07 13:35:22 -07:00
Tobi
9e0fdf969e Merge pull request #1318 from AudricV/yt_fix-subs-count-channels-with-pronouns 
[YouTube] Fix parsing subscribers count of channels with pronouns
 2025-07-07 12:32:43 -07:00
Tobi
c4dcef4d51 Merge pull request #1319 from AudricV/peertube_support-video-only-streams-properly 
[PeerTube] Detect video-only streams
 2025-07-07 12:26:27 -07:00