Commons:Bots/Requests

Shortcut: COM:BRFA

If you want to run a bot on Commons, you must get permission first. To do so, file a request following the instructions below.

Please read Commons:Bots before making a request for bot permission.

I	Create a user account (while logged in to your normal account) and user page for the bot On the bot's userpage, add {{Bot}}, which automatically adds the page to Category:Commons bots. Then add the following information to the bot's userpage (all this is mandatory): Operator: Who the creator/operator is and how they can best be contacted Tasks: Details of the bot's task or tasks Operation: Whether the bot is manually assisted or runs automatically When: When it operates (continuously, intermittently, or at specified intervals) Maximum edit rate: The bot's maximum edit rate (eg edits per minute) Language: The language and/or program that it is running
II	Create your bot request: Replace "YourBotName" with your bot's username in the box below and click the button. Complete the questions on the resulting page and save it. Remember that your bot will be allowed to operate only for those tasks that you have specified in your request. Add your bot request to the list here: Edit the following request list, adding the following text to the top of the appropriate section (replacing "YourBotName" with your bot's name): {{Commons:Bots/Requests/YourBotName}}
III	Test run You can be demanded to make a short test run with your bot account (30–50 edits/uploads) to allow other users to review your bot's tasks. Unauthorized test run is not allowed.
IV	Waiting for approval. You now need to wait for community approval. A bureaucrat will close the request and will also grant a bot flag, where necessary. Closed requests are moved to Commons:Bots/Archive.

Archives List all Requests Requests by closing date

Requests made on this page are automatically transcluded in Commons:Requests and votes for wider comment.

Requests for permission to run a bot

watch

Before making a bot request, please read the new version of the Commons:Bots page. Read Commons:Bots#Information on bots and make sure you have added the required details to the bot's page. A good example can be found here.

When complete, pages listed here should be archived to Commons:Bots/Archive.

Any user may comment on the merits of the request to run a bot. Please give reasons, as that makes it easier for the closing bureaucrat. Read Commons:Bots before commenting.

YannBot (talk · contribs)

Operator: Yann (talk · contributions · Statistics · Recent activity · block log · User rights log · uploads · Global account information) (assign permissions)

Bot's tasks for which permission is being sought: Uploading files from archives. I will start by about a few thousands files. Then I'd like to continue Fae's work started here: Commons:IA books.

Automatic or manually assisted: supervised

Edit type (e.g. Continuous, daily, one time run): one batch {~100} of files at a time

Maximum edit rate (e.g. edits per minute): 2

Bot flag requested: (Y/N): Y

Programming language(s): Python (Pywikibot)

Yann (talk) 21:07, 25 May 2023 (UTC)

Discussion

I uploaded about 70 files for testing. Yann (talk) 21:15, 25 May 2023 (UTC)

It'll be good idea to fill structured data. See request below as example. --EugeneZelenko (talk) 14:23, 26 May 2023 (UTC)

Isn't there already a bot doing that? For the files uploaded as test, they will one WD entry for the book. No need of a bot for that. Yann (talk) 15:59, 26 May 2023 (UTC)

DifoolBot (talk · contribs)

Operator: Difool (talk · contributions · Statistics · Recent activity · block log · User rights log · uploads · Global account information) (assign permissions)

Bot's tasks for which permission is being sought: Add Structured data to files from the Rijksmuseum Amsterdam, specifically the files in these five categories:

1. Pen and ink drawings in the Rijksmuseum Amsterdam
2. Postcards in the Rijksmuseum Amsterdam
3. Tinted lithographs in the Rijksmuseum Amsterdam
4. Wood engravings in the Rijksmuseum Amsterdam
5. Engravings in the Rijksmuseum Amsterdam

These are about 20,000 files.

Automatic or manually assisted: Automatic

Edit type (e.g. Continuous, daily, one time run): Intermittently, daily

Maximum edit rate (e.g. edits per minute): Default setting, so about six edits per minute max.

Bot flag requested: (Y/N): Yes

Programming language(s): Python, pywikipediabot.

Difool (talk) 14:11, 23 May 2023 (UTC)

Discussion

• Please make test run on several actual files from different categories. --EugeneZelenko (talk) 14:51, 23 May 2023 (UTC)

  I've changed 10 files. I reverted one file because it was renamed, something I hadn't taken into account. Difool (talk) 03:45, 26 May 2023 (UTC)

• Looks OK for me, but please take a look on duplicated instance of in File:Kinderspelen Prenten-magazijn voor de jeugd (serietitel op object) Kinderspelen (serietitel op object), RP-P-OB-206.377.jpg and File:Brakende man in bed, RP-P-1938-2030.jpg. --EugeneZelenko (talk) 14:20, 26 May 2023 (UTC)

WLKBot (talk · contribs)

Operator: WLKBot (talk · contributions · Statistics · Recent activity · block log · User rights log · uploads · Global account information) (assign permissions) User:Kim Bach is the operator, he can be contacted through commons mail, User:MSanderhoff can also be contacted.

Bot's tasks for which permission is being sought: Upload of public domain images of artwork and related metadata, using the Template:Artwork of artwork provided through partnerships with Danish GLAMS.

The first partnership is with SMK - The National Gallery of Denmark a project page is created for the purpose Commons:SMK - Statens Museum for Kunst

For the SMK contribution, the images are downloaded and metadata is mapped to the Artwork template, utilising the Statens Museum for Kunst, SMK API: entry KMS1 and the wikilabskultur Artwork template preprocessor.

Automatic or manually assisted: The bot is manually assisted and will most likely be running from a stand alone computer

Edit type (e.g. Continuous, daily, one time run): One time run (several batches)

When: The bot operates at specified intervals (batches). Activity depends on the availability of new batches from GLAMs, it can be started on demand by the local operator, on a stand alone pc

Maximum edit rate (e.g. edits per minute): Most likely 10-50 edits per minute, but only in short bursts, and it can be set to a much lower rate. The first proposed upload will be in the form of approx. 500 files of 10-30MB in size, time to finish is not of the essence, the total size of the contribution from SMK currently is approx. 70.000 images, the collection is expected to be quite static.

Bot flag requested: (Y/N): Y

Programming language(s): The bot is written in Python using the pywikibot library

The source code is available on GitHub

WLKBot (talk) 17:13, 16 December 2021 (UTC)

Discussion

• Please make test run. --EugeneZelenko (talk) 15:46, 17 December 2021 (UTC)

First test run, a total of 25 images uploaded. --WLKBot (talk) 09:44, 20 December 2021 (UTC)/User:Kim Bach

Could medium be internationalized with template or Wikidata item? If not, please use language tag. Same for object type like Blyant. Could bot add painting by or more detailed category? Please also make batch categories hidden. --EugeneZelenko (talk) 16:04, 20 December 2021 (UTC)

• Example File:Christen Købke, Lille søstudie med en ø i den venezianske lagune, 1838, KMS7020, Statens Museum for Kunst.jpg.

This file should have a default sortkey (Defaultsort) "Købke, Christen, 1838" as we generally sort items by the creator's last name.

It should have a category of "Paintings by Christen Købke in Statens Museum for Kunst" with a local sortkey of "|1838" (after the category name). I did that manually.

It should have a category of "Paintings by Christen Købke" with a local sortkey of "|1838" (after the category name). Is this not redundant? No, because those paintings will eventually be divided up as landscapes, marine paintings, portraits and so on.

It might have a category of "19th-century paintings in the Statens Museum for Kunst". Note: Here it is the Statens Museum for Kunst. Don't ask.

It might have a category of "Landscape paintings in the Statens Museum for Kunst".

It might have a category of "1838 paintings from Denmark".

Happy programming. Cheers Rsteen (talk) 12:54, 21 December 2021 (UTC)

Thank you for your comment, and the detailed example, it's very helpful, I can definitely improve on the categorisation along the lines you suggested WLKBot (talk) 21:00, 22 December 2021 (UTC)

@Kim Bach: please don't use your bot account to make manual edits like the previous comment.

• You're adding {{Statens Museum for Kunst collaboration project}}, so you shouldn't be adding Category:Images released under the CC0 1.0 Universal license by Statens Museum for Kunst
• Please wrap the medium field in {{Da}} to make clear that this is in Danish. Makes it easier to later replace in internationalization.
• I don't care that much about the categorization as long a you add some object type categories it should be fine.
• Including the copyright and license template makes {{Artwork}} really big. Please create two sections example and put the permission part in the bottom section.
• Please only use {{Creator}} if the template exists (or you're creating it) because otherwise we'll end up with broken links for years
• Not sure if you plan to upload some 3D works, but in that case you can use {{Licensed-PD}}
• Bonus points if you also add structured data ([https://github.com/multichill/toollabs/blob/master/bot/commons/wikidata_uploader.py some example code)

I enjoyed visiting the SMK and looking forward to having all this content on Commons! Multichill (talk) 18:29, 1 January 2022 (UTC)

@Kim Bach and MSanderhoff:  ? --Krd 04:26, 5 February 2022 (UTC)

@Krd I assume that the "?" means that you'd like an update :-). We're busy implementing the changes suggested by the community, and we will not run another test until we're confident in the changes we're making, this should only be weeks away. Kim Bach (talk) 13:59, 6 February 2022 (UTC)

Please report current status and intentions. --Krd 15:12, 28 November 2022 (UTC)

@Krd

Current status is that we've addressed most of the suggestions by EugeneZelenko and Multichill and some of the suggestions by RSteen

This is a list of the changes we've made

• medium has been wrapped in language template and will be using Wikidata items in test and production runs
• object type has been wrapped in language template and will be using Wikidata items in in test and production runs
• painting/... by categories added
• batch categories made hidden
• No longer adds Category:Images released under the CC0 1.0 Universal license by Statens Museum for Kunst
• Wraps the medium field in {{Da}}, will move to Wikidata items in test and production runs
• We'll be adding structured data using QuickStatements after we've uploaded the images, we will add the Wikidata-item to the Artwork template when we have created the Wikidata-items or identified already existing Wikidata-items
• We've created two sections for the copyright and permissions
• We're no longer using Creator template, we've switched to using Wikidata items, and will not upload images that has no Wikidata item for creators. The idea is to add the ones that are missing to Wikidata as we go along, using QuickStatements
• We've added ...by... categories, but we think that most of the might have suggestions by Rsteen are better handled through structured data, which we'll be adding using QuickStatements

Below is an example of a file that was uploaded manually, where the Wikitext was generated by our updated code, and used to improve the code, we'll perform a few more of these before doing a test run, but we feel like we're ready

Giovanni Battista Piranesi, Det indre af Pantheon, 1768, KKSgb9860-86, Statens Museum for Kunst

The intention is to resume testing, firstly by doing some more manual uploads, in the context of Kim Bach, in preparation of new test runs.

Pending approval we'll move slowly to production runs, the plan is to do that one creator at a time.

--Kim Bach (talk) 05:32, 30 November 2022 (UTC)

Please feel free to do a small test run. Krd 05:59, 30 November 2022 (UTC)

@Krd We've made a test run of 20 new media files. We're of course referencing a number of new categories, we should probably create those as well. Maybe as we go along, maybe at a later point in time.

We're referencing a number of object types that aren't created yet. It looks like this is done in LUA code. I do have all the Q item numbers of the object types we're using on hand, but we're also planning on adding structure data using QuickStatemenst, we're currently testing that.

Regards --Kim Bach (talk) 10:05, 3 December 2022 (UTC)

Since you (EugeneZelenko, Multichill and RSteen) commented on our first test run last year, I’d like you to know that we’re completed our second test run, and that we”ve tried to address most of your suggestions. —Kim Bach (talk) 09:17, 6 December 2022 (UTC)

object type should be internationalized. Also excessive indentation for license tags and newlines between license tags and categories. --EugeneZelenko (talk) 16:10, 6 December 2022 (UTC)

Had a quick look, nice images! My points

• I see a lot of redundant white space. Please trim a bit more,
• I'm not a huge fan of extra information fields. What's the point of adding "SMK record created", "SMK record modified" and the (broken) iiif link?
• You're getting a warning because you are using {{PD-old}}. I guess most works are covered by {{PD-old-100-expired}}?
• In the upload edit summary I would put something like "uploaded artwork from https://open.smk.dk/en/artwork/image/KKSgb22228" instead of "created artwork"

Multichill (talk) 18:09, 6 December 2022 (UTC)

Thank you Multichill, I've incorporated your suggestions and I'll be omitting the "other fields" in the future, they were meant for internal bookkeeping (created and updated timestamps of metadata from SMK)

I've updated the wikitext for this image, trying to take your and Eugene's suggestions into account

Tobias Stimmer, Romerne indtager Satricum, 1574, KKSgb22345, Statens Museum for Kunst

Kim Bach (talk) 22:22, 6 December 2022 (UTC)

Thank you EugeneZelenko, I can address your comments in this way:

Internationalisation of object type

I suppose you mean that I should use this:

• object type={{en|Woodcut print}}

Excessive white space

I can beautify the wikitext along these guidelines:

1. No leading whitespace
2. No spaces between equal signs
3. No double line feeds

For instance:

{{Artwork

|artist=<Artist Name>

|title=<The Title>

|description=<The Description>

...

}} Kim Bach (talk) 19:03, 6 December 2022 (UTC)

On my opinion spaces after pipes make text more readable. --EugeneZelenko (talk) 15:39, 7 December 2022 (UTC)

Agree, bit too much trimming see Template:Artwork#Usage for a good example how it should look. Multichill (talk) 18:18, 7 December 2022 (UTC)

Thanks, I’ll go for that, is at ends with the suggestion by Eugene (space after pipe, I guess that’s not to importerne.

Another possible issue is the new lines. They flush left, and it doesn”t look good. Is there a way to add indentation that doesn’t show up in the rendered wiki page, I suppose the List templates could be used for that purpose. Kim Bach (talk) 14:40, 8 December 2022 (UTC)

I've made changes that tries to accommodate your standard usage suggestion - this is a "bit" confusing because the Usage sample and the Multilingual sample differs :-/ - and the suggestion with spaces trailing pipes by EugeneZelenko. I've updated the wikitext for the Tobias Stimmer image to reflect the suggested changes :-) Kim Bach (talk) 11:09, 9 December 2022 (UTC)

Title consists from two tags. Is it possible to merge title in two sentences in one tag? Also there is d:Q18219090, so will be good idea to use it instead of text. --EugeneZelenko (talk) 15:54, 9 December 2022 (UTC)

Multiple titles

The reason we have two titles for this particular item, is that the museum DB can have several titles for an item.

The best thing to do, might be to limit it to one - since one of the titles is considered official by the museum. We could use that, and then, at a later point in time, add more titles to wikidata, we could also change it to one line, like you suggested.

Using wikidata

Wikidata, works fine for the medium tag wrapped in the Technique template, we can do that, since we've already mapped the Q-numbers.

Is it possible to use Q-numbers for the object type too?

We've tried it, with no luck in nailing the syntax. Kim Bach (talk) 22:24, 9 December 2022 (UTC)

Looks like Module:Artwork should be enhanced. --EugeneZelenko (talk) 15:17, 10 December 2022 (UTC)

Hi. Take a look at this upload File:Elisabeth Jerichau Baumann, En såret dansk kriger, 1865, KMS852, Statens Museum for Kunst.jpg. The dimensions are not ok. They are in mm instead of cm. Do not know if this is a general error, and have not seen any comments on it before. Cheers Rsteen (talk) 10:35, 20 December 2022 (UTC)

Thank you, yes, that was a general error in the code from last year, everything was a factor 10 off, as you noticed, we've fixed that since. I forgot to fix this manually in the batch from last year, so I'll do that. Kim Bach (talk) 15:57, 21 December 2022 (UTC)

An updated status. We're currently refactoring the category generating code, this will most likely not be ready until january 2023. We still welcome comments to the last test run. Kim Bach (talk) 16:10, 21 December 2022 (UTC)

@Krd

We've refactored the code, and are now ready to perform a new test. The main change is that we now try to create new category pages, if they don't exist Kim Bach (talk) 21:31, 11 January 2023 (UTC)

@Multichill and EugeneZelenko we're ready to perform a new test run :-) --Kim Bach (talk) 14:31, 13 January 2023 (UTC)

Just to clarify: was Module:Artwork improved meanwhile? --EugeneZelenko (talk) 15:59, 13 January 2023 (UTC)

No, it looks like the procedure involves requesting an edit, I wasn't aware of that, I've added a comment on the talk page. Kim Bach (talk) 20:31, 13 January 2023 (UTC)

Can anybody please summarize what is the current status and what exactly is missing? --Krd 06:41, 29 January 2023 (UTC)

Indeed, as you might have noticed from the discussion above, what is missing is a (correct) mapping of the "object type" field.

We've been in discussion with the maintainers of the artwork template, unfortunately the field is not supporting Wikidata Q-items directly, so we'll have to add these to the mapping table, all are P31-tems.

Once we've done that - a few hundered - we're ready for at new test run.

I'll update you on the progress later this week. Kim Bach (talk) 14:11, 31 January 2023 (UTC)

I've drafted a version of the object type data table, where I've added the object types I was missing, you can find it here User:Kim_Bach/sandbox/object_type_data. I've asked the maintainers of the Artwork template to review it.

Kim Bach (talk) 17:15, 4 February 2023 (UTC)

Multichill can you help me by reviewing my suggested changes to the mapping table as drafted here User:Kim_Bach/sandbox/object_type_data. —Kim Bach (talk) 22:21, 9 February 2023 (UTC)

@Krd:

I've opened a discussion here I need help with review and approval of suggested changes to Module:I18n/objects/data and here I need help with review and approval of suggested changes to Module:I18n/objects/data

Kim Bach (talk) 08:02, 8 March 2023 (UTC)

Sadly I have no idea how to put this forward. Any additional ideas welcome. Krd 11:25, 6 April 2023 (UTC)

Thank you, I had some good suggestions from the community and I will work along those lines. I’ll post an update when the changes are made. Kim Bach (talk) 10:00, 7 April 2023 (UTC)

@Kim Bach: sorry, I completely lost track of this. I thought we had some minor points left and the bot was ready to go. I updated the module. Is that the last thing? Maybe do some test edits now as a final check? Multichill (talk) 16:39, 8 April 2023 (UTC)

Thank you, and that is understandable, it has been a long process, and I have been very cautious. I just had help from MGA73 and I'm compiling a new list, that isn't such a mess when trying to apply a diff. Kim Bach (talk) 12:44, 10 April 2023 (UTC)

@Kim Bach: . The module has been updated so hopefully it should include the things you suggested earlier. --MGA73 (talk) 13:04, 10 April 2023 (UTC)

Once again thank you for the great effort, I'll check and add items that might be missing, but we should be good to go! Kim Bach (talk) 13:18, 10 April 2023 (UTC)

I've checked the updated list, and all the additions I wanted are indeed present! Kim Bach (talk) 14:09, 10 April 2023 (UTC)

@Kim Bach: Great. Then you could perhaps upload some files for final test? --MGA73 (talk) 17:54, 11 April 2023 (UTC)

Thanks, I'll do that next week, I'm currently busy doing volunteer work for DepressionsForeningen :-) Kim Bach (talk) 11:35, 23 April 2023 (UTC)

I've just completed a test run with 20 images Kim Bach (talk) 20:53, 27 April 2023 (UTC)

Test of 2023-04-27 There was an issue with the "object type" property, I've corrected the error, and updated the new images, so they should reflect the changes in the code--Kim Bach (talk) 07:21, 28 April 2023 (UTC)

@Kim Bach: I see that you're using {{PD-old-auto-expired}}, but not setting deathyear. Either fill it or just use {{PD-old-100-expired}} directly to correct this. Multichill (talk) 15:12, 28 April 2023 (UTC)

I've updated the code, and I'm ready for a new test run Kim Bach (talk) 18:50, 29 April 2023 (UTC)

If you assign categories automatically, you need to follow the general naming convention for categories more closely. Take File:Tobias Stimmer, Gaius Duillius sejrer i et søslag over kartaginienserne, 1574, KKSgb22331, Statens Museum for Kunst.jpg. It is categorized as Woodcut prints by Tobias Stimmer in the Statens Museum for Kunst. The normal category would be Woodcuts by Tobias Stimmer in the Statens Museum for Kunst. It is also categorized as Graphics by Tobias Stimmer in the Statens Museum for Kunst. That category is not necessary and represents overcategorization. The category Woodcuts by Tobias Stimmer in the Statens Museum for Kunst should lead to Woodcuts in the Statens Museum for Kunst, which again should lead to Prints in the Statens Museum for Kunst. Try and take a look at how it is done at the Rijksmuseum, which is more or less the gold standard. Take Category:Prints in the Rijksmuseum Amsterdam. Here all the different types of prints are gathered and you can go to subcategories by artist, by century and so on. We should attempt something similar. Cheers Rsteen (talk) 11:51, 1 May 2023 (UTC)

Thank you. @Rsteen: What we need to do, is to look much closer at the category mapping, as you suggested, it might not be that diffocult. Do you understand JSON format? Kim Bach (talk) 04:01, 2 May 2023 (UTC)

Absolutely. And thank you for your work. There is a goldmine of artworks at Statens Museum for Kunst. Cheers Rsteen (talk) 05:05, 2 May 2023 (UTC)

So...The strategy would be to create categories similar to the ones the Rijksmuseum Amsterdam has created, and then create a mapping table between those categories and the data returned by SMKAPI. Taking into consideration that some existing categories are not using the naming convention. I'll propose something.

I've created a navigation box based on the Category:Rijksmuseum Amsterdam the draft is located here User:Kim Bach/sandbox/Statens Museum for Kunst, it should be a template instead.

Statens Museum for Kunst uses a total of 128 unique categories, Rijksmuseum Amsterdam has 38 main categories in their navigation box. I will work on some kind of consolidation, like the 38 categories in the navigation box. I'm sure the art historians at SMK can be helpful in grouping the categories. Kim Bach (talk) 11:50, 3 May 2023 (UTC)

I have attempted to create the category hierarchy needed for the woodcut prints, the file I've updated File:Tobias Stimmer, Gaius Duillius sejrer i et søslag over kartaginienserne, 1574, KKSgb22331, Statens Museum for Kunst.jpg has been updated to use the new categories.

I'd like you to help with a review before I write the code to achieve this.

These are the categories I've created (phew - and this is just a start - dates, decades, centuries etc. still are missing):

•  N Category:Works in the Statens Museum for Kunst by artist‎ 14:16  +141‎  ‎Kim Bach diskussion bidrag‎ (created category)
•  N Category:Collections of the Statens Museum for Kunst by object type‎ 14:09  +137‎  ‎Kim Bach diskussion bidrag‎ (created category)
•  N Category:Woodcut prints by artist by museum‎ 14:01  +216‎  ‎Kim Bach diskussion bidrag‎ (created category)
•  N Category:Woodcut Prints by artist by museum‎ 13:54  +176‎  ‎Kim Bach diskussion bidrag‎ (created category)
•  N Category:Woodcut prints by artist‎ 13:50  +308‎  ‎Kim Bach diskussion bidrag‎ (created category)
•  N Category:Collections of woodcut prints‎ 12:55  +120‎  ‎Kim Bach diskussion bidrag‎ (created category)
•  N Category:Woodcut prints by museum‎ 12:53  +186‎  ‎Kim Bach diskussion bidrag‎ (created category)
•  N Category:Woodcut prints by museum by country‎ 12:51  +208‎  ‎Kim Bach diskussion bidrag‎ (created category)
•  N Category:Categories of Denmark by location‎ 12:44  +167‎  ‎Kim Bach diskussion bidrag‎ (created category)
•  N Category:Woodcut prints in Denmark by location‎ 12:43  +95‎  ‎Kim Bach diskussion bidrag‎ (created category)
•  N Category:Woodcut prints in Denmark by museum‎ 12:39  +305‎  ‎Kim Bach diskussion bidrag‎ (created category)
•  N Category:Woodcut prints in Denmark‎ 12:35  +215‎  ‎Kim Bach diskussion bidrag‎ (created category)
•  N Category:Woodcut prints in Copenhagen‎ 12:24  +137‎  ‎Kim Bach diskussion bidrag‎ (created category)
•  m Template:SMKNavBox‎ 10:32  −7‎  ‎Kim Bach diskussion bidrag‎ (added image)

Kim Bach (talk) 14:29, 5 May 2023 (UTC)

Thank you for your kind assistance, I can see that I'm creating a lot of work for you. I'll be more careful before updating in production in the future. Kim Bach (talk) 21:36, 5 May 2023 (UTC)