Merge dfd94f4f2e into 4782a67038

Closes issue 4880

.github/workflows/build-stable-container.yml

@@ -1,6 +1,7 @@
 name: Build and release container
 
 on:
+  workflow_dispatch:
   push:
     tags:
       - "v*"
@@ -46,9 +47,11 @@ jobs:
         uses: docker/metadata-action@v5
         with:
           images: quay.io/invidious/invidious
+          flavor: |
+            latest=false
           tags: |
             type=semver,pattern={{version}}
-            type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', 'master') }}
+            type=raw,value=latest
           labels: |
             quay.expires-after=12w
 
@@ -70,10 +73,11 @@ jobs:
         with:
           images: quay.io/invidious/invidious
           flavor: |
+            latest=false
             suffix=-arm64
           tags: |
             type=semver,pattern={{version}}
-            type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', 'master') }}
+            type=raw,value=latest
           labels: |
             quay.expires-after=12w
 

.gitignore

@@ -7,3 +7,4 @@
 /invidious
 /sentry
 /config/config.yml
+/invidious-videojs-dep-install

CHANGELOG.md

@@ -1,6 +1,189 @@
 # CHANGELOG
 
-## 2024-04-26
+
+## v2.20240825.2 (2024-08-26)
+
+This releases fixes the container tags pushed on quay.io.
+Previously, the ARM64 build was released under the `latest` tag, instead of `latest-arm64`.
+
+### Full list of pull requests merged since the last release (newest first)
+
+CI: Fix docker container tags ([#4883], by @SamantazFox)
+
+[#4877]: https://github.com/iv-org/invidious/pull/4877
+
+
+## v2.20240825.1 (2024-08-25)
+
+Add patch component to be [semver] compliant and make github actions happy.
+
+[semver]: https://semver.org/
+
+### Full list of pull requests merged since the last release (newest first)
+
+Allow manual trigger of release-container build ([#4877], thanks @syeopite)
+
+[#4877]: https://github.com/iv-org/invidious/pull/4877
+
+
+## v2.20240825.0 (2024-08-25)
+
+### New features & important changes
+
+#### For users
+
+* The search bar now has a button that you can click!
+* Youtube URLs can be pasted directly in the search bar. Prepend search query with a
+  backslash (`\`) to disable that feature (useful if you need to search for a video whose
+  title contains some youtube URL).
+* On the channel page the "streams" tab can be sorted by either: "newest", "oldest" or "popular"
+* Lots of translations have been updated (thanks to our contributors on Weblate!)
+* Videos embedded in local HTML files (e.g: a webpage saved from a blog) can now be played
+
+#### For instance owners
+
+* Invidious now has the ability to provide a `po_token` and `visitordata` to Youtube in order to
+  circumvent current Youtube restrictions.
+* Invidious can use an (optional) external signature server like [inv_sig_helper]. Please note that
+  some videos can't be played without that signature server.
+* The Helm charts were moved to a separate repo: https://github.com/iv-org/invidious-helm-chart
+* We have changed how containers are released: the `latest` tag now tracks tagged releases, whereas
+  the `master` tag tracks the most recent commits of the `master` branch ("nightly" builds).
+
+[inv_sig_helper]: https://github.com/iv-org/inv_sig_helper
+
+#### For developpers
+
+* The versions of Crystal that we test in CI/CD are now: `1.9.2`, `1.10.1`, `1.11.2`, `1.12.1`.
+  Please note that due to a bug in the `libxml` bindings (See [#4256]), versions prior to `1.10.0`
+  are not recommended to use.
+* Thanks to @syeopite, the code is now [ameba] compliant.
+* Ameba is part of our CI/CD pipeline, and its rules will be enforced in future PRs.
+* The transcript code has been rewritten to permit transcripts as a feature rather than being
+  only a workaround for captions. Trancripts feature is coming soon!
+* Various fixes regarding the logic interacting with Youtube
+* The `sort_by` parameter can be used on the `/api/v1/channels/{id}/streams` endpoint. Accepted
+  values are: "newest", "oldest" and "popular"
+
+[ameba]: https://github.com/crystal-ameba/ameba
+[#4256]: https://github.com/iv-org/invidious/issues/4256
+
+
+### Bugs fixed
+
+#### User-side
+
+* Channels: fixed broken "subscribers" and "views" counters
+* Watch page: playback position is reset at the end of a video, so that the next time this video
+  is watched, it will start from the beginning rather than 15 seconds before the end
+* Watch page: the items in the "add to playlist" drop down are now sorted alphabetically
+* Videos: the "genre" URL is now always pointing to a valid webpage
+* Playlists: Fixed `Could not parse N episodes` error on podcast playlists
+* All external links should now have the [`rel`] attibute set to `noreferrer noopener` for
+  increased privacy.
+* Preferences: Fixed the admin-only "modified source code" input being ignored
+* Watch/channel pages: use the full image URL in `og:image` and `twitter:image` meta tags
+
+[`rel`]: https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes/rel
+
+#### API
+
+* fixed the `local` parameter not applying to `formatStreams` on `/api/v1/videos/{id}`
+* fixed an `Index out of bounds` error hapenning when a playlist had no videos
+* fixed duplicated query parameters in proxied video URLs
+* Return actual video height/width/fps rather than hard coded values
+* Fixed the `/api/v1/popular` endpoint not returning a proper error code/message when the
+  popular page/endpoint are disabled.
+
+
+### Full list of pull requests merged since the last release (newest first)
+
+* HTML: Sort playlists alphabetically in watch page drop down ([#4853], by @SamantazFox)
+* Videos: Fix XSS vulnerability in description/comments ([#4852], thanks _anonymous_)
+* YtAPI: Bump client versions ([#4849], by @SamantazFox)
+* SigHelper: Fix inverted time comparison in 'check_update' ([#4845], by @SamantazFox)
+* Storyboards: Various fixes and code cleaning ([#4153], by SamantazFox)
+* Fix lint errors introduced in #4146 and #4295 ([#4876], thanks @syeopite)
+* Search: Add support for Youtube URLs ([#4146], by @SamantazFox)
+* Channel: Render age restricted channels ([#4295], thanks @ChunkyProgrammer)
+* Ameba: Miscellaneous fixes ([#4807], thanks @syeopite)
+* API: Proxy formatStreams URLs too ([#4859], thanks @colinleroy)
+* UI: Add search button to search bar ([#4706], thanks @thansk)
+* Add ability to set po_token and visitordata ID ([#4789], thanks @unixfox)
+* Add support for an external signature server ([#4772], by @SamantazFox)
+* Ameba: Fix Naming/VariableNames ([#4790], thanks @syeopite)
+* Translations update from Hosted Weblate ([#4659])
+* Ameba: Fix Lint/UselessAssign ([#4795], thanks @syeopite)
+* HTML: Add rel="noreferrer noopener" to external links ([#4667], thanks @ulmemxpoc)
+* Remove unused methods in Invidious::LogHandler ([#4812], thanks @syeopite)
+* Ameba: Fix Lint/NotNilAfterNoBang ([#4796], thanks @syeopite)
+* Ameba: Fix unused argument Lint warnings ([#4805], thanks @syeopite)
+* Ameba: i18next.cr fixes ([#4806], thanks @syeopite)
+* Ameba: Disable rules ([#4792], thanks @syeopite)
+* Channel: parse subscriber count and channel banner ([#4785], thanks @ChunkyProgrammer)
+* Player: Fix playback position of already watched videos ([#4731], thanks @Fijxu)
+* Videos: Fix genre url being unusable ([#4717], thanks @meatball133)
+* API: Fix out of bound error on empty playlists ([#4696], thanks @Fijxu)
+* Handle playlists cataloged as Podcast ([#4695], thanks @Fijxu)
+* API: Fix duplicated query parameters in proxied video URLs ([#4587], thanks @absidue)
+* API: Return actual stream height, width and fps ([#4586], thanks @absidue)
+* Preferences: Fix handling of modified source code URL ([#4437], thanks @nooptek)
+* API: Fix URL for vtt subtitles ([#4221], thanks @karelrooted)
+* Channels: Add sort options to streams ([#4224], thanks @src-tinkerer)
+* API: Fix error code for disabled popular endpoint ([#4296], thanks @iBicha)
+* Allow embedding videos in local HTML files ([#4450], thanks @tomasz1986)
+* CI: Bump Crystal version matrix ([#4654], by @SamantazFox)
+* YtAPI: Remove API keys like official clients ([#4655], by @SamantazFox)
+* HTML: Use full URL in the og:image property ([#4675], thanks @Fijxu)
+* Rewrite transcript logic to be more generic ([#4747], thanks @syeopite)
+* CI: Run Ameba ([#4753], thanks @syeopite)
+* CI: Add release based containers ([#4763], thanks @syeopite)
+* move helm chart to a dedicated github repository ([#4711], thanks @unixfox)
+
+[#4146]: https://github.com/iv-org/invidious/pull/4146
+[#4153]: https://github.com/iv-org/invidious/pull/4153
+[#4221]: https://github.com/iv-org/invidious/pull/4221
+[#4224]: https://github.com/iv-org/invidious/pull/4224
+[#4295]: https://github.com/iv-org/invidious/pull/4295
+[#4296]: https://github.com/iv-org/invidious/pull/4296
+[#4437]: https://github.com/iv-org/invidious/pull/4437
+[#4450]: https://github.com/iv-org/invidious/pull/4450
+[#4586]: https://github.com/iv-org/invidious/pull/4586
+[#4587]: https://github.com/iv-org/invidious/pull/4587
+[#4654]: https://github.com/iv-org/invidious/pull/4654
+[#4655]: https://github.com/iv-org/invidious/pull/4655
+[#4659]: https://github.com/iv-org/invidious/pull/4659
+[#4667]: https://github.com/iv-org/invidious/pull/4667
+[#4675]: https://github.com/iv-org/invidious/pull/4675
+[#4695]: https://github.com/iv-org/invidious/pull/4695
+[#4696]: https://github.com/iv-org/invidious/pull/4696
+[#4706]: https://github.com/iv-org/invidious/pull/4706
+[#4711]: https://github.com/iv-org/invidious/pull/4711
+[#4717]: https://github.com/iv-org/invidious/pull/4717
+[#4731]: https://github.com/iv-org/invidious/pull/4731
+[#4747]: https://github.com/iv-org/invidious/pull/4747
+[#4753]: https://github.com/iv-org/invidious/pull/4753
+[#4763]: https://github.com/iv-org/invidious/pull/4763
+[#4772]: https://github.com/iv-org/invidious/pull/4772
+[#4785]: https://github.com/iv-org/invidious/pull/4785
+[#4789]: https://github.com/iv-org/invidious/pull/4789
+[#4790]: https://github.com/iv-org/invidious/pull/4790
+[#4792]: https://github.com/iv-org/invidious/pull/4792
+[#4795]: https://github.com/iv-org/invidious/pull/4795
+[#4796]: https://github.com/iv-org/invidious/pull/4796
+[#4805]: https://github.com/iv-org/invidious/pull/4805
+[#4806]: https://github.com/iv-org/invidious/pull/4806
+[#4807]: https://github.com/iv-org/invidious/pull/4807
+[#4812]: https://github.com/iv-org/invidious/pull/4812
+[#4845]: https://github.com/iv-org/invidious/pull/4845
+[#4849]: https://github.com/iv-org/invidious/pull/4849
+[#4852]: https://github.com/iv-org/invidious/pull/4852
+[#4853]: https://github.com/iv-org/invidious/pull/4853
+[#4859]: https://github.com/iv-org/invidious/pull/4859
+[#4876]: https://github.com/iv-org/invidious/pull/4876
+
+
+## v2.20240427 (2024-04-27)
 
 Major bug fixes:
  * Videos: Use android test suite client (#4650, thanks @SamantazFox)

scripts/fetch-player-dependencies.cr

@@ -1,14 +1,253 @@
 require "http"
 require "yaml"
+require "file_utils"
 require "digest/sha1"
 require "option_parser"
 require "colorize"
 
+# Represents an "install_instruction" section specified per dependency in `videojs-dependencies.yml`
+#
+# This is used to modify the download logic for dependencies that are packaged differently.
+struct InstallInstruction
+  include YAML::Serializable
+
+  property js_path : String? = nil
+  property css_path : String? = nil
+  property download_as : String? = nil
+  property no_styling : Bool = false
+end
+
+# Object representing a dependency specified within `videojs-dependencies.yml`
+class ConfigDependency
+  include YAML::Serializable
+
+  property version : String
+  property shasum : String
+
+  property install_instructions : InstallInstruction? = nil
+
+  # Checks if the current dependency needs to be installed/updated
+  def fetch?(name : String)
+    path = "assets/videojs/#{name}"
+
+    # Check for missing dependency files
+    #
+    # Does the directory exist?
+    # Does the Javascript file exist?
+    # Does the CSS file exist?
+    if !Dir.exists?(path)
+      Dir.mkdir(path)
+      return true
+    elsif !(File.exists?("#{path}/#{name}.js") || File.exists?("#{path}/versions.yml"))
+      return true
+    elsif !(self.install_instructions.try &.no_styling) && !File.exists?("#{path}/#{name}.css")
+      return true
+    end
+
+    # Check if we need to update the dependency
+
+    versions = File.open("#{path}/versions.yml") do |file|
+      YAML.parse(file).as_h
+    end
+
+    if versions["version"].as_s != self.version || versions["minified"].as_bool != CONFIG.minified
+      # Clear directory
+      {"*.js", "*.css"}.each do |file_types|
+        Dir.glob("#{path}/#{file_types}").each do |file_path|
+          File.delete(file_path)
+        end
+      end
+
+      return true
+    end
+
+    return false
+  end
+end
+
+# Object representing the `videojs-dependencies.yml` file
+class PlayerDependenciesConfig
+  include YAML::Serializable
+
+  property version : String
+  property registry_url : String
+  property cache_directory : String
+  property dependencies : Hash(YAML::Any, ConfigDependency)
+
+  def get_dependencies_to_fetch
+    return self.dependencies.select { |name, config| config.fetch?(name.as_s) }
+  end
+end
+
+# Runtime Dependency config for easy access to all the variables
+class Config
+  property minified : Bool
+  property skip_checksum : Bool
+  property clear_cache : Bool
+
+  property dependency_config : PlayerDependenciesConfig
+
+  def initialize(path : String)
+    @minified = false
+    @skip_checksum = false
+    @clear_cache = false
+
+    @dependency_config = PlayerDependenciesConfig.from_yaml(File.read(path))
+  end
+
+  # Less verbose way to access @dependency_config.registry_url
+  def registry_url
+    return @dependency_config.registry_url
+  end
+
+  # Less verbose way to access @dependency_config.cache_directory
+  def cache_directory
+    return @dependency_config.cache_directory
+  end
+end
+
+# Object representing a player dependency
+class Dependency
+  @config : ConfigDependency
+
+  def initialize(@config : ConfigDependency, @dependency : String)
+    @download_path = "#{CONFIG.cache_directory}/#{@dependency}"
+    @destination_path = "assets/videojs/#{@dependency}"
+  end
+
+  private def validate_checksum(io)
+    return if CONFIG.skip_checksum
+
+    digest = Digest::SHA1.hexdigest(io)
+    if digest != @config.shasum
+      raise IO::Error.new("Checksum for '#{@dependency}' failed. \"#{digest}\" does not match configured \"#{@config.shasum}\"")
+    end
+  end
+
+  # Requests and downloads a specific dependency from NPM
+  #
+  # Validates a cached tarball if it already exists.
+  private def request_dependency
+    downloaded_package_path = "#{@download_path}/package.tgz"
+
+    # Create a download directory for the dependency if it does not already exist
+    if Dir.exists?(@download_path)
+      # Validate checksum of existing cached tarball
+      # Fetches a new one when the checksum fails.
+      if File.exists?(downloaded_package_path)
+        begin
+          return self.validate_checksum(File.open(downloaded_package_path))
+        rescue IO::Error
+        end
+      end
+    else
+      Dir.mkdir(@download_path)
+    end
+
+    HTTP::Client.get("#{CONFIG.registry_url}/#{@dependency}/-/#{@dependency}-#{@config.version}.tgz") do |response|
+      data = response.body_io.gets_to_end
+      File.write(downloaded_package_path, data)
+      self.validate_checksum(data)
+    end
+  end
+
+  # Moves a VideoJS dependency file of the given extension from extracted tarball to Invidious directory
+  private def move_file(full_target_path, extension)
+    minified_target_path = sprintf(full_target_path, {"file_extension": ".min.#{extension}"})
+
+    if CONFIG.minified && File.exists?(minified_target_path)
+      target_path = minified_target_path
+    else
+      target_path = sprintf(full_target_path, {"file_extension": ".#{extension}"})
+    end
+
+    if download_as = @config.install_instructions.try &.download_as
+      destination_path = "#{@destination_path}/#{sprintf(download_as, {"file_extension": ".#{extension}"})}"
+    else
+      destination_path = @destination_path
+    end
+
+    FileUtils.cp(target_path, destination_path)
+  end
+
+  # Fetch path of where a VideoJS dependency is located in the extracted tarball
+  private def fetch_path(is_css)
+    if is_css
+      raw_target_path = @config.install_instructions.try &.css_path
+    else
+      raw_target_path = @config.install_instructions.try &.js_path
+    end
+
+    if raw_target_path
+      return "#{@download_path}/package/#{raw_target_path}"
+    else
+      return "#{@download_path}/package/dist/#{@dependency}%{file_extension}"
+    end
+  end
+
+  # Wrapper around `#move_file` to move the dependency's JS file
+  private def move_js_file
+    return self.move_file(self.fetch_path(is_css: false), "js")
+  end
+
+  # Wrapper around `#move_file` to move the dependency's CSS file
+  #
+  # Does nothing with the CSS file does not exist.
+  private def move_css_file
+    path = self.fetch_path(is_css: true)
+
+    if File.exists?(sprintf(path, {"file_extension": ".css"}))
+      return move_file(path, "css")
+    end
+  end
+
+  # Updates the dependency's versions.yml with the current fetched version and its minified status
+  private def update_versions_yaml
+    File.open("#{@destination_path}/versions.yml", "w") do |io|
+      YAML.build(io) do |builder|
+        builder.mapping do
+          # Versions
+          builder.scalar "version"
+          builder.scalar "#{@config.version}"
+
+          builder.scalar "minified"
+          builder.scalar CONFIG.minified
+        end
+      end
+    end
+  end
+
+  # Installs a VideoJS dependency into Invidious
+  def install
+    self.request_dependency
+
+    # Crystal's stdlib provides no way of extracting a tarball
+    `tar -vzxf '#{@download_path}/package.tgz' -C '#{@download_path}'`
+    raise "Extraction for #{@dependency} failed" if !$?.success?
+
+    self.move_js_file
+    self.move_css_file
+
+    self.update_versions_yaml
+  end
+end
+
+CONFIG = Config.new("videojs-dependencies.yml")
+
+# Hacky solution to get separated arguments when called from invidious.cr
+if ARGV.size == 1
+  parser_args = [] of String
+  ARGV[0].split(",") { |str| parser_args << str.strip }
+else
+  parser_args = ARGV
+end
+
 # Taken from https://crystal-lang.org/api/1.1.1/OptionParser.html
-minified = false
-OptionParser.parse do |parser|
+OptionParser.parse(parser_args) do |parser|
   parser.banner = "Usage: Fetch VideoJS dependencies [arguments]"
-  parser.on("-m", "--minified", "Use minified versions of VideoJS dependencies (performance and bandwidth benefit)") { minified = true }
+  parser.on("-m", "--minified", "Use minified versions of VideoJS dependencies (performance and bandwidth benefit)") { CONFIG.minified = true }
+  parser.on("--skip-checksum", "Skips the checksum validation of downloaded files") { CONFIG.skip_checksum = true }
+  parser.on("--clear-cache", "Clears the cache and re-downloads all dependency files") { CONFIG.clear_cache = true }
 
   parser.on("-h", "--help", "Show this help") do
     puts parser
@@ -22,124 +261,17 @@ OptionParser.parse do |parser|
   end
 end
 
-required_dependencies = File.open("videojs-dependencies.yml") do |file|
-  YAML.parse(file).as_h
-end
-
-def update_versions_yaml(required_dependencies, minified, dep_name)
-  File.open("assets/videojs/#{dep_name}/versions.yml", "w") do |io|
-    YAML.build(io) do |builder|
-      builder.mapping do
-        # Versions
-        builder.scalar "version"
-        builder.scalar "#{required_dependencies[dep_name]["version"]}"
-
-        builder.scalar "minified"
-        builder.scalar minified
-      end
-    end
-  end
-end
-
-# The first step is to check which dependencies we'll need to install.
-# If the version we have requested in `videojs-dependencies.yml` is the
-# same as what we've installed, we shouldn't do anything. Likewise, if it's
-# different or the requested dependency just isn't present, then it needs to be
-# installed.
-
-# Since we can't know when videojs-youtube-annotations is updated, we'll just always fetch
-# a new copy each time.
-dependencies_to_install = [] of String
-
-required_dependencies.keys.each do |dep|
-  dep = dep.to_s
-  path = "assets/videojs/#{dep}"
-  # Check for missing dependencies
-  if !Dir.exists?(path)
-    Dir.mkdir(path)
-    dependencies_to_install << dep
-  else
-    config = File.open("#{path}/versions.yml") do |file|
-      YAML.parse(file).as_h
-    end
-
-    if config["version"].as_s != required_dependencies[dep]["version"].as_s || config["minified"].as_bool != minified
-      `rm -rf #{path}/*.js #{path}/*.css`
-      dependencies_to_install << dep
-    end
-  end
-end
-
-# Now we begin the fun part of installing the dependencies.
-# But first we'll setup a temp directory to store the plugins
-tmp_dir_path = "#{Dir.tempdir}/invidious-videojs-dep-install"
-Dir.mkdir(tmp_dir_path) if !Dir.exists? tmp_dir_path
+# Create cache directory
+Dir.mkdir(CONFIG.cache_directory) if !Dir.exists? CONFIG.cache_directory
 
+dependencies_to_install = CONFIG.dependency_config.get_dependencies_to_fetch
 channel = Channel(String | Exception).new
 
-dependencies_to_install.each do |dep|
+dependencies_to_install.each do |dep_name, dependency_config|
   spawn do
-    dep_name = dep
-    download_path = "#{tmp_dir_path}/#{dep}"
-    dest_path = "assets/videojs/#{dep}"
-
-    HTTP::Client.get("https://registry.npmjs.org/#{dep}/-/#{dep}-#{required_dependencies[dep]["version"]}.tgz") do |response|
-      Dir.mkdir(download_path)
-      data = response.body_io.gets_to_end
-      File.write("#{download_path}/package.tgz", data)
-
-      # https://github.com/iv-org/invidious/pull/2397#issuecomment-922375908
-      if `sha1sum #{download_path}/package.tgz`.split(" ")[0] != required_dependencies[dep]["shasum"]
-        raise Exception.new("Checksum for '#{dep}' failed")
-      end
-    end
-
-    # Unless we install an external dependency, crystal provides no way of extracting a tarball.
-    # Thus we'll go ahead and call a system command.
-    `tar -vzxf '#{download_path}/package.tgz' -C '#{download_path}'`
-    raise "Extraction for #{dep} failed" if !$?.success?
-
-    # Would use File.rename in the following steps but for some reason it just doesn't work here.
-    # Video.js itself is structured slightly differently
-    dep = "video" if dep == "video.js"
-
-    # This dep nests everything under an additional JS or CSS folder
-    if dep == "silvermine-videojs-quality-selector"
-      js_path = "js/"
-
-      # It also stores their quality selector as `quality-selector.css`
-      `mv #{download_path}/package/dist/css/quality-selector.css #{dest_path}/quality-selector.css`
-    else
-      js_path = ""
-    end
-
-    # Would use File.rename but for some reason it just doesn't work here.
-    if minified && File.exists?("#{download_path}/package/dist/#{js_path}#{dep}.min.js")
-      `mv #{download_path}/package/dist/#{js_path}#{dep}.min.js #{dest_path}/#{dep}.js`
-    else
-      `mv #{download_path}/package/dist/#{js_path}#{dep}.js #{dest_path}/#{dep}.js`
-    end
-
-    # Fetch CSS which isn't guaranteed to exist
-    #
-    # Also, video JS changes structure here once again...
-    dep = "video-js" if dep == "video"
-
-    # VideoJS marker uses a dot on the CSS files.
-    dep = "videojs.markers" if dep == "videojs-markers"
-
-    if File.exists?("#{download_path}/package/dist/#{dep}.css")
-      if minified && File.exists?("#{download_path}/package/dist/#{dep}.min.css")
-        `mv #{download_path}/package/dist/#{dep}.min.css #{dest_path}/#{dep}.css`
-      else
-        `mv #{download_path}/package/dist/#{dep}.css #{dest_path}/#{dep}.css`
-      end
-    end
-
-    # Update/create versions file for the dependency
-    update_versions_yaml(required_dependencies, minified, dep_name)
-
-    channel.send(dep_name)
+    dependency = Dependency.new(dependency_config, dep_name.as_s)
+    dependency.install
+    channel.send(dep_name.as_s)
   rescue ex
     channel.send(ex)
   end
@@ -161,4 +293,6 @@ else
 end
 
 # Cleanup
-`rm -rf #{tmp_dir_path}`
+if CONFIG.clear_cache
+  FileUtils.rm_r("#{CONFIG.cache_directory}")
+end

src/invidious.cr

@@ -144,12 +144,26 @@ Invidious::Database.check_integrity(CONFIG)
   # Running the script by itself would show some colorful feedback while this doesn't.
   # Perhaps we should just move the script to runtime in order to get that feedback?
 
-  {% puts "\nChecking player dependencies, this may take more than 20 minutes... If it is stuck, check your internet connection.\n" %}
-  {% if flag?(:minified_player_dependencies) %}
-    {% puts run("../scripts/fetch-player-dependencies.cr", "--minified").stringify %}
-  {% else %}
-    {% puts run("../scripts/fetch-player-dependencies.cr").stringify %}
+  {% fetch_script_arguments = [] of StringLiteral %}
+
+  {%
+    potential_arguments = {
+      {:minified_player_dependencies, "--minified"},
+      {:skip_player_dependencies_checksum, "--skip-checksum"},
+      {:clear_player_dependencies_cache, "--clear-cache"},
+    }
+  %}
+
+  {% for potential_argument in potential_arguments %}
+    {% flag_, script_argument = potential_argument %}
+
+    {% if flag?(flag_) %}
+      {% fetch_script_arguments << script_argument.id %}
+    {% end %}
   {% end %}
+
+  {% puts "\nChecking player dependencies, this may take more than 20 minutes... If it is stuck, check your internet connection.\n" %}
+  {% puts run("../scripts/fetch-player-dependencies.cr", fetch_script_arguments.splat).stringify %}
   {% puts "\nDone checking player dependencies, now compiling Invidious...\n" %}
 {% end %}
 

src/invidious/views/components/player_sources.ecr

@@ -1,12 +1,12 @@
-<link rel="stylesheet" href="/videojs/video.js/video-js.css?v=<%= ASSET_COMMIT %>">
+<link rel="stylesheet" href="/videojs/video.js/video.js.css?v=<%= ASSET_COMMIT %>">
 <link rel="stylesheet" href="/videojs/videojs-http-source-selector/videojs-http-source-selector.css?v=<%= ASSET_COMMIT %>">
-<link rel="stylesheet" href="/videojs/videojs-markers/videojs.markers.css?v=<%= ASSET_COMMIT %>">
+<link rel="stylesheet" href="/videojs/videojs-markers/videojs-markers.css?v=<%= ASSET_COMMIT %>">
 <link rel="stylesheet" href="/videojs/videojs-share/videojs-share.css?v=<%= ASSET_COMMIT %>">
 <link rel="stylesheet" href="/videojs/videojs-vtt-thumbnails/videojs-vtt-thumbnails.css?v=<%= ASSET_COMMIT %>">
 <link rel="stylesheet" href="/videojs/videojs-mobile-ui/videojs-mobile-ui.css?v=<%= ASSET_COMMIT %>">
 <link rel="stylesheet" href="/css/player.css?v=<%= ASSET_COMMIT %>">
 
-<script src="/videojs/video.js/video.js?v=<%= ASSET_COMMIT %>"></script>
+<script src="/videojs/video.js/video.js.js?v=<%= ASSET_COMMIT %>"></script>
 <script src="/videojs/videojs-mobile-ui/videojs-mobile-ui.js?v=<%= ASSET_COMMIT %>"></script>
 <script src="/videojs/videojs-contrib-quality-levels/videojs-contrib-quality-levels.js?v=<%= ASSET_COMMIT %>"></script>
 <script src="/videojs/videojs-http-source-selector/videojs-http-source-selector.js?v=<%= ASSET_COMMIT %>"></script>

videojs-dependencies.yml

@@ -1,54 +1,80 @@
-# Due to a 'video append of' error (see #3011), we're stuck on 7.12.1.
-video.js:
-  version: 7.12.1
-  shasum: 1d12eeb1f52e3679e8e4c987d9b9eb37e2247fa2
+version: 1
+registry_url: "https://registry.npmjs.org"
 
-videojs-contrib-quality-levels:
-  version: 2.1.0
-  shasum: 046e9e21ed01043f512b83a1916001d552457083
+# Dependencies are stored as <dependency_name>/package.tgz
+cache_directory: "./invidious-videojs-dep-install"
 
-videojs-http-source-selector:
-  version: 1.1.6
-  shasum: 073aadbea0106ba6c98d6b611094dbf8554ffa1f
+dependencies:
+  #Due to a 'video append of' error (see #3011), we're stuck on 7.12.1.
+  video.js:
+    version: 7.12.1
+    shasum: 1d12eeb1f52e3679e8e4c987d9b9eb37e2247fa2
 
-videojs-markers:
-  version: 1.0.1
-  shasum: d7f8d804253fd587813271f8db308a22b9f7df34
+    install_instructions:
+      js_path: "dist/video%{file_extension}"
+      css_path: "dist/video-js%{file_extension}"
 
-videojs-mobile-ui:
-  version: 0.6.1
-  shasum: 0e146c4c481cbee0729cb5e162e558b455562cd0
+      # Normalize names to simplify File.exists? check
+      download_as: "video.js%{file_extension}"
 
-videojs-overlay:
-  version: 2.1.4
-  shasum: 5a103b25374dbb753eb87960d8360c2e8f39cc05
+  videojs-contrib-quality-levels:
+    version: 2.1.0
+    shasum: 046e9e21ed01043f512b83a1916001d552457083
 
-videojs-share:
-  version: 3.2.1
-  shasum: 0a3024b981387b9d21c058c829760a72c14b8ceb
+    install_instructions:
+      no_styling: true
 
-videojs-vr:
-  version: 1.8.0
-  shasum: 7f2f07f760d8a329c615acd316e49da6ee8edd34
+  videojs-http-source-selector:
+    version: 1.1.6
+    shasum: 073aadbea0106ba6c98d6b611094dbf8554ffa1f
 
-videojs-vtt-thumbnails:
-  version: 0.0.13
-  shasum: d1e7d47f4ed80bb52f5fc4f4bad4bfc871f5970f
+  videojs-markers:
+    version: 1.0.1
+    shasum: d7f8d804253fd587813271f8db308a22b9f7df34
 
-# We're using iv-org's fork of videojs-quality-selector,
-# which isn't published on NPM, and doesn't have any
-# easy way of fetching the compiled variant.
-#
-# silvermine-videojs-quality-selector:
-#   version: 1.1.2
-#   shasum: 94033ff9ee52ba6da1263b97c9a74d5b3dfdf711
+    install_instructions:
+      css_path: "dist/videojs.markers%{file_extension}"
+      download_as: "videojs-markers%{file_extension}"
+
+  videojs-mobile-ui:
+    version: 0.6.1
+    shasum: 0e146c4c481cbee0729cb5e162e558b455562cd0
+
+  videojs-overlay:
+    version: 2.1.4
+    shasum: 5a103b25374dbb753eb87960d8360c2e8f39cc05
+
+  videojs-share:
+    version: 3.2.1
+    shasum: 0a3024b981387b9d21c058c829760a72c14b8ceb
+
+  videojs-vr:
+    version: 1.8.0
+    shasum: 7f2f07f760d8a329c615acd316e49da6ee8edd34
+
+  videojs-vtt-thumbnails:
+    version: 0.0.13
+    shasum: d1e7d47f4ed80bb52f5fc4f4bad4bfc871f5970f
+
+  # We're using iv-org's fork of videojs-quality-selector,
+  # which isn't published on NPM, and doesn't have any
+  # easy way of fetching the compiled variant.
+
+  # silvermine-videojs-quality-selector:
+  #   version: 1.1.2
+  #   shasum: 94033ff9ee52ba6da1263b97c9a74d5b3dfdf711
+
+  #   install_instructions:
+  #     js_path: "dist/js/silvermine-videojs-quality-selector%{file_extension}"
+  #     css_path: "dist/css/quality-selector%{file_extension}"
+  #     download_as: silvermine-videojs-quality-selector%{file_extension}
 
 
-# Ditto. Although this extension contains the complied variant in its git repo, 
-# it lacks any sort of versioning. As such, the script will ignore it.
-#
-# videojs-youtube-annotations:
-#   github: https://github.com/afrmtbl/videojs-youtube-annotations
+  # Ditto. Although this extension contains the complied variant in its git repo,
+  # it lacks any sort of versioning. As such, the script will ignore it.
+  #
+  # videojs-youtube-annotations:
+  #   github: https://github.com/afrmtbl/videojs-youtube-annotations