Literate programming

[foxyseta]

Literate programming languages, such as Literate Haskell, Literate Agda, and Literate Coffeescript, are currently marked with their literate extensions (e.g., .lagda). This in incorrect: Linguist does not support multiple extension segments (see #1438), and these extensions are meant to precede the extension of the markup language/typesetting sytem hosting them. (e.g., .lagda.typ). Because of this Linguist limit, file extension analysis couldn't possibly work.

[lildude]

Because of this Linguist limit, file extension analysis couldn't possibly work.

Have you looked at the code and tested this? 😉

Linguist does not support multiple extension segments (see #1438)

You are referencing a PR that is over 10 years old. Things change in that time including the fact that linguist does support extensions with multiple segments as you can see in the code here:

linguist/lib/linguist/blob.rb

Lines 61 to 73 in 1647b5b

	# Public: Return an array of the file extensions
	#
	# >> Linguist::Blob.new("app/views/things/index.html.erb").extensions
	# => [".html.erb", ".erb"]
	#
	# Returns an Array
	def extensions
	_, *segments = name.downcase.split(".", -1)
	segments.map.with_index do |segment, index|
	"." + segments[index..-1].join(".")
	end
	end

… and can also see from the many instances in the languages.yml file which work without issue.

The only language Linguist know about that has .lagda anywhere in the extension is Literate Agda (which makes sense as it's l + agda):

linguist/lib/linguist/languages.yml

Lines 3981 to 3989 in 1647b5b

	Literate Agda:
	type: programming
	color: "#315665"
	group: Agda
	extensions:
	- ".lagda"
	tm_scope: none
	ace_mode: text
	language_id: 205

… which was added many many years ago. As this is the only language with that extension, all instances will be identified as that language. In the case of your .lagda.typ extension (and other such examples), the match will happen only on the last segment, .typ, which is currently shared by Typst and XML.

You can see all the other literate languages linguist supports and their extensions in the languages.yml file immediately after the Literate Agda entry referenced above.

This all brings us to the real question: what brought you to this incorrect conclusion and prompted you to start this discussion? What problem are you seeing?

[foxyseta]

Have you looked at the code and tested this? 😉

Thanks for pointing me to the right code line. Is there no user manual or auto-generated API docs on this? It would have been somewhat time-consuming for me to browse the code-base as a non-Ruby user. My problem was *.agda.typ files being detected as Typst rather than Literate Agda, which is consistent with languages.yml (first thing I did was take a look there). Similar behavior for the other literate languages. At this point, I would propose changes to languages.yml.

[lildude]

Is there no user manual or auto-generated API docs on this?

No. Not anything more than what’s in the docs directory. Generally this is sufficient for most as this library is primarily used by GitHub and that’s how users “interact” with it.

I would propose changes to languages.yml.

Feel free to submit a PR. You can find details in the CONTRIBUTING.md file. Note that we have minimum usage requirements for PRs to be accepted.

If you don’t want to or can’t submit a PR, you can implement an override in your repos to force the language.

[foxyseta]

Thanks! Current usage seems to be:

Extension	Results
.lagda.tex	179
.lagda.rst	632
.lagda.md	8.9k
.lagda.typ	0
.lagda.org	24
.lagda.tree	5

The contributing page claims the ">=200" usage requirement is a "preference" for "most cases". However, all these file extensions are documented on the same official Agda documentation page, so adding only some of them would feel like half-baking this to me. Also, I am personally here for the unpopular .lagda.typ. What shall I do?

I am currenlty on go-enry, which fetches language extensions from linguist, and I am still trying to figure out why git-overriding for Literate Agda in particular does not work with that (this is clearly not a Linguist problem: I just wanted to explain why overriding wouldn't cut it for me atm).

[lildude]

What shall I do?

Nothing. None of the extensions are popular enough for inclusion. We also only add extensions as they meet requirements, even if they're official and formally documented.

... trying to figure out why git-overriding

I'm pretty sure go-enry doesn't support git overrides at all. Check their list of limitations.

[foxyseta]

None of the extensions are popular enough for inclusion.

Don't .lagda.rst and .lagda.md meet the requirements? Or is it more like, "they meet the requirements but are not really popular enough actually".

I'm pretty sure go-enry doesn't support git overrides at all. Check their list of limitations.

Yeah, that's why this is implemented at forge level by ForgeJo. I asked in that community.

[lildude]

Whoops. I misread the .lagda.md total. Yes, it would but from a quick search and read of the docs you linked to I don't see there would be any benefit to adding this to Literate Agda as it appears this is standard markdown so GitHub renders them as such. GitHub doesn't interpret code in codeblocks so changing the language of these files will break the rendering people are used to and expecting.

.lagda.rst would not as these don't appear to be files one would reasonably expect to only occur once per repo thus they need to meet the 2000 files requirement.