Skip to content

Commit 78d6908

Browse files
MichaIngMichaIng
committed
Change X-Robots-Tag header from "none" to "noindex, nofollow"
While "none" is indeed equivalent to "noindex, nofollow" for Google, but seems to be not supported by Bing and probably other search engines. https://developer.mozilla.org/en-US/docs/Web/HTML/Element/meta/name#other_metadata_names https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag?hl=de#comma-separated-list https://www.bing.com/webmasters/help/which-robots-metatags-does-bing-support-5198d240 Signed-off-by: MichaIng <[email protected]>
1 parent 9160b37 commit 78d6908

12 files changed

Lines changed: 36 additions & 36 deletions

File tree

‎.htaccess‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@
3131
Header always set X-Permitted-Cross-Domain-Policies "none"
3232

3333
Header onsuccess unset X-Robots-Tag
34-
Header always set X-Robots-Tag "none"
34+
Header always set X-Robots-Tag "noindex, nofollow"
3535

3636
Header onsuccess unset X-XSS-Protection
3737
Header always set X-XSS-Protection "1; mode=block"

‎build/integration/features/carddav.feature‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ Feature: carddav
4646
|X-Content-Type-Options |nosniff|
4747
|X-Frame-Options|SAMEORIGIN|
4848
|X-Permitted-Cross-Domain-Policies|none|
49-
|X-Robots-Tag|none|
49+
|X-Robots-Tag|noindex, nofollow|
5050
|X-XSS-Protection|1; mode=block|
5151

5252
Scenario: Exporting the picture of ones own contact
@@ -60,5 +60,5 @@ Feature: carddav
6060
|X-Content-Type-Options |nosniff|
6161
|X-Frame-Options|SAMEORIGIN|
6262
|X-Permitted-Cross-Domain-Policies|none|
63-
|X-Robots-Tag|none|
63+
|X-Robots-Tag|noindex, nofollow|
6464
|X-XSS-Protection|1; mode=block|

‎build/integration/features/dav-v2.feature‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ Feature: dav-v2
2727
|X-Content-Type-Options |nosniff|
2828
|X-Frame-Options|SAMEORIGIN|
2929
|X-Permitted-Cross-Domain-Policies|none|
30-
|X-Robots-Tag|none|
30+
|X-Robots-Tag|noindex, nofollow|
3131
|X-XSS-Protection|1; mode=block|
3232
And Downloaded content should start with "Welcome to your Nextcloud account!"
3333

‎build/integration/features/webdav-related.feature‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ Feature: webdav-related
5353
And User "user0" moves file "/textfile0.txt" to "/testshare/textfile0.txt"
5454
And the HTTP status code should be "403"
5555
When Downloading file "/testshare/textfile0.txt"
56-
Then the HTTP status code should be "404"
56+
Then the HTTP status code should be "404"
5757

5858
Scenario: Moving a file to overwrite a file in a folder with no permissions
5959
Given using old dav path
@@ -251,7 +251,7 @@ Feature: webdav-related
251251
|X-Content-Type-Options |nosniff|
252252
|X-Frame-Options|SAMEORIGIN|
253253
|X-Permitted-Cross-Domain-Policies|none|
254-
|X-Robots-Tag|none|
254+
|X-Robots-Tag|noindex, nofollow|
255255
|X-XSS-Protection|1; mode=block|
256256
And Downloaded content should start with "Welcome to your Nextcloud account!"
257257

‎core/js/setupchecks.js‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -628,7 +628,7 @@
628628
if (xhr.status === 200) {
629629
var securityHeaders = {
630630
'X-Content-Type-Options': ['nosniff'],
631-
'X-Robots-Tag': ['none'],
631+
'X-Robots-Tag': ['noindex, nofollow'],
632632
'X-Frame-Options': ['SAMEORIGIN', 'DENY'],
633633
'X-Permitted-Cross-Domain-Policies': ['none'],
634634
};

‎core/js/tests/specs/setupchecksSpec.js‎

Lines changed: 23 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1569,7 +1569,7 @@ describe('OC.SetupChecks tests', function() {
15691569
msg: 'The "X-Content-Type-Options" HTTP header is not set to "nosniff". This is a potential security or privacy risk, as it is recommended to adjust this setting accordingly.',
15701570
type: OC.SetupChecks.MESSAGE_TYPE_WARNING
15711571
}, {
1572-
msg: 'The "X-Robots-Tag" HTTP header is not set to "none". This is a potential security or privacy risk, as it is recommended to adjust this setting accordingly.',
1572+
msg: 'The "X-Robots-Tag" HTTP header is not set to "noindex, nofollow". This is a potential security or privacy risk, as it is recommended to adjust this setting accordingly.',
15731573
type: OC.SetupChecks.MESSAGE_TYPE_WARNING
15741574
}, {
15751575
msg: 'The "X-Frame-Options" HTTP header is not set to "SAMEORIGIN". This is a potential security or privacy risk, as it is recommended to adjust this setting accordingly.',
@@ -1596,7 +1596,7 @@ describe('OC.SetupChecks tests', function() {
15961596
suite.server.requests[0].respond(
15971597
200,
15981598
{
1599-
'X-Robots-Tag': 'none',
1599+
'X-Robots-Tag': 'noindex, nofollow',
16001600
'X-Frame-Options': 'SAMEORIGIN',
16011601
'Strict-Transport-Security': 'max-age=15768000;preload',
16021602
'X-Permitted-Cross-Domain-Policies': 'none',
@@ -1627,7 +1627,7 @@ describe('OC.SetupChecks tests', function() {
16271627
{
16281628
'X-XSS-Protection': '1; mode=block',
16291629
'X-Content-Type-Options': 'nosniff',
1630-
'X-Robots-Tag': 'none',
1630+
'X-Robots-Tag': 'noindex, nofollow',
16311631
'X-Frame-Options': 'SAMEORIGIN',
16321632
'Strict-Transport-Security': 'max-age=15768000',
16331633
'X-Permitted-Cross-Domain-Policies': 'none',
@@ -1650,7 +1650,7 @@ describe('OC.SetupChecks tests', function() {
16501650
'Strict-Transport-Security': 'max-age=15768000',
16511651
'X-XSS-Protection': '1; mode=block; report=https://example.com',
16521652
'X-Content-Type-Options': 'nosniff',
1653-
'X-Robots-Tag': 'none',
1653+
'X-Robots-Tag': 'noindex, nofollow',
16541654
'X-Frame-Options': 'SAMEORIGIN',
16551655
'X-Permitted-Cross-Domain-Policies': 'none',
16561656
'Referrer-Policy': 'no-referrer',
@@ -1670,7 +1670,7 @@ describe('OC.SetupChecks tests', function() {
16701670
'Strict-Transport-Security': 'max-age=15768000',
16711671
'X-XSS-Protection': '1; mode=block',
16721672
'X-Content-Type-Options': 'nosniff',
1673-
'X-Robots-Tag': 'none',
1673+
'X-Robots-Tag': 'noindex, nofollow',
16741674
'X-Frame-Options': 'SAMEORIGIN',
16751675
'X-Permitted-Cross-Domain-Policies': 'none',
16761676
'Referrer-Policy': 'no-referrer',
@@ -1690,7 +1690,7 @@ describe('OC.SetupChecks tests', function() {
16901690
'Strict-Transport-Security': 'max-age=15768000',
16911691
'X-XSS-Protection': '1',
16921692
'X-Content-Type-Options': 'nosniff',
1693-
'X-Robots-Tag': 'none',
1693+
'X-Robots-Tag': 'noindex, nofollow',
16941694
'X-Frame-Options': 'SAMEORIGIN',
16951695
'X-Permitted-Cross-Domain-Policies': 'none',
16961696
'Referrer-Policy': 'no-referrer',
@@ -1715,7 +1715,7 @@ describe('OC.SetupChecks tests', function() {
17151715
'Strict-Transport-Security': 'max-age=15768000',
17161716
'X-XSS-Protection': '0',
17171717
'X-Content-Type-Options': 'nosniff',
1718-
'X-Robots-Tag': 'none',
1718+
'X-Robots-Tag': 'noindex, nofollow',
17191719
'X-Frame-Options': 'SAMEORIGIN',
17201720
'X-Permitted-Cross-Domain-Policies': 'none',
17211721
'Referrer-Policy': 'no-referrer',
@@ -1742,7 +1742,7 @@ describe('OC.SetupChecks tests', function() {
17421742
'Strict-Transport-Security': 'max-age=15768000',
17431743
'X-XSS-Protection': '1; mode=block',
17441744
'X-Content-Type-Options': 'nosniff',
1745-
'X-Robots-Tag': 'none',
1745+
'X-Robots-Tag': 'noindex, nofollow',
17461746
'X-Frame-Options': 'SAMEORIGIN',
17471747
'X-Permitted-Cross-Domain-Policies': 'none',
17481748
'Referrer-Policy': 'no-referrer',
@@ -1762,7 +1762,7 @@ describe('OC.SetupChecks tests', function() {
17621762
'Strict-Transport-Security': 'max-age=15768000',
17631763
'X-XSS-Protection': '1; mode=block',
17641764
'X-Content-Type-Options': 'nosniff',
1765-
'X-Robots-Tag': 'none',
1765+
'X-Robots-Tag': 'noindex, nofollow',
17661766
'X-Frame-Options': 'SAMEORIGIN',
17671767
'X-Permitted-Cross-Domain-Policies': 'none',
17681768
'Referrer-Policy': 'no-referrer-when-downgrade',
@@ -1782,7 +1782,7 @@ describe('OC.SetupChecks tests', function() {
17821782
'Strict-Transport-Security': 'max-age=15768000',
17831783
'X-XSS-Protection': '1; mode=block',
17841784
'X-Content-Type-Options': 'nosniff',
1785-
'X-Robots-Tag': 'none',
1785+
'X-Robots-Tag': 'noindex, nofollow',
17861786
'X-Frame-Options': 'SAMEORIGIN',
17871787
'X-Permitted-Cross-Domain-Policies': 'none',
17881788
'Referrer-Policy': 'strict-origin',
@@ -1802,7 +1802,7 @@ describe('OC.SetupChecks tests', function() {
18021802
'Strict-Transport-Security': 'max-age=15768000',
18031803
'X-XSS-Protection': '1; mode=block',
18041804
'X-Content-Type-Options': 'nosniff',
1805-
'X-Robots-Tag': 'none',
1805+
'X-Robots-Tag': 'noindex, nofollow',
18061806
'X-Frame-Options': 'SAMEORIGIN',
18071807
'X-Permitted-Cross-Domain-Policies': 'none',
18081808
'Referrer-Policy': 'strict-origin-when-cross-origin',
@@ -1822,7 +1822,7 @@ describe('OC.SetupChecks tests', function() {
18221822
'Strict-Transport-Security': 'max-age=15768000',
18231823
'X-XSS-Protection': '1; mode=block',
18241824
'X-Content-Type-Options': 'nosniff',
1825-
'X-Robots-Tag': 'none',
1825+
'X-Robots-Tag': 'noindex, nofollow',
18261826
'X-Frame-Options': 'SAMEORIGIN',
18271827
'X-Permitted-Cross-Domain-Policies': 'none',
18281828
'Referrer-Policy': 'same-origin',
@@ -1842,7 +1842,7 @@ describe('OC.SetupChecks tests', function() {
18421842
'Strict-Transport-Security': 'max-age=15768000',
18431843
'X-XSS-Protection': '1; mode=block',
18441844
'X-Content-Type-Options': 'nosniff',
1845-
'X-Robots-Tag': 'none',
1845+
'X-Robots-Tag': 'noindex, nofollow',
18461846
'X-Frame-Options': 'SAMEORIGIN',
18471847
'X-Permitted-Cross-Domain-Policies': 'none',
18481848
'Referrer-Policy': 'origin',
@@ -1867,7 +1867,7 @@ describe('OC.SetupChecks tests', function() {
18671867
'Strict-Transport-Security': 'max-age=15768000',
18681868
'X-XSS-Protection': '1; mode=block',
18691869
'X-Content-Type-Options': 'nosniff',
1870-
'X-Robots-Tag': 'none',
1870+
'X-Robots-Tag': 'noindex, nofollow',
18711871
'X-Frame-Options': 'SAMEORIGIN',
18721872
'X-Permitted-Cross-Domain-Policies': 'none',
18731873
'Referrer-Policy': 'origin-when-cross-origin',
@@ -1892,7 +1892,7 @@ describe('OC.SetupChecks tests', function() {
18921892
'Strict-Transport-Security': 'max-age=15768000',
18931893
'X-XSS-Protection': '1; mode=block',
18941894
'X-Content-Type-Options': 'nosniff',
1895-
'X-Robots-Tag': 'none',
1895+
'X-Robots-Tag': 'noindex, nofollow',
18961896
'X-Frame-Options': 'SAMEORIGIN',
18971897
'X-Permitted-Cross-Domain-Policies': 'none',
18981898
'Referrer-Policy': 'unsafe-url',
@@ -1919,7 +1919,7 @@ describe('OC.SetupChecks tests', function() {
19191919
{
19201920
'X-XSS-Protection': '1; mode=block',
19211921
'X-Content-Type-Options': 'nosniff',
1922-
'X-Robots-Tag': 'none',
1922+
'X-Robots-Tag': 'noindex, nofollow',
19231923
'X-Frame-Options': 'SAMEORIGIN',
19241924
'X-Permitted-Cross-Domain-Policies': 'none',
19251925
'Referrer-Policy': 'no-referrer',
@@ -1965,7 +1965,7 @@ describe('OC.SetupChecks tests', function() {
19651965
{
19661966
'X-XSS-Protection': '1; mode=block',
19671967
'X-Content-Type-Options': 'nosniff',
1968-
'X-Robots-Tag': 'none',
1968+
'X-Robots-Tag': 'noindex, nofollow',
19691969
'X-Frame-Options': 'SAMEORIGIN',
19701970
'X-Permitted-Cross-Domain-Policies': 'none',
19711971
'Referrer-Policy': 'no-referrer',
@@ -1990,7 +1990,7 @@ describe('OC.SetupChecks tests', function() {
19901990
'Strict-Transport-Security': 'max-age=15551999',
19911991
'X-XSS-Protection': '1; mode=block',
19921992
'X-Content-Type-Options': 'nosniff',
1993-
'X-Robots-Tag': 'none',
1993+
'X-Robots-Tag': 'noindex, nofollow',
19941994
'X-Frame-Options': 'SAMEORIGIN',
19951995
'X-Permitted-Cross-Domain-Policies': 'none',
19961996
'Referrer-Policy': 'no-referrer',
@@ -2015,7 +2015,7 @@ describe('OC.SetupChecks tests', function() {
20152015
'Strict-Transport-Security': 'iAmABogusHeader342',
20162016
'X-XSS-Protection': '1; mode=block',
20172017
'X-Content-Type-Options': 'nosniff',
2018-
'X-Robots-Tag': 'none',
2018+
'X-Robots-Tag': 'noindex, nofollow',
20192019
'X-Frame-Options': 'SAMEORIGIN',
20202020
'X-Permitted-Cross-Domain-Policies': 'none',
20212021
'Referrer-Policy': 'no-referrer',
@@ -2039,7 +2039,7 @@ describe('OC.SetupChecks tests', function() {
20392039
'Strict-Transport-Security': 'max-age=15768000',
20402040
'X-XSS-Protection': '1; mode=block',
20412041
'X-Content-Type-Options': 'nosniff',
2042-
'X-Robots-Tag': 'none',
2042+
'X-Robots-Tag': 'noindex, nofollow',
20432043
'X-Frame-Options': 'SAMEORIGIN',
20442044
'X-Permitted-Cross-Domain-Policies': 'none',
20452045
'Referrer-Policy': 'no-referrer',
@@ -2059,7 +2059,7 @@ describe('OC.SetupChecks tests', function() {
20592059
'Strict-Transport-Security': 'max-age=99999999',
20602060
'X-XSS-Protection': '1; mode=block',
20612061
'X-Content-Type-Options': 'nosniff',
2062-
'X-Robots-Tag': 'none',
2062+
'X-Robots-Tag': 'noindex, nofollow',
20632063
'X-Frame-Options': 'SAMEORIGIN',
20642064
'X-Permitted-Cross-Domain-Policies': 'none',
20652065
'Referrer-Policy': 'no-referrer',
@@ -2079,7 +2079,7 @@ describe('OC.SetupChecks tests', function() {
20792079
'Strict-Transport-Security': 'max-age=99999999; includeSubDomains',
20802080
'X-XSS-Protection': '1; mode=block',
20812081
'X-Content-Type-Options': 'nosniff',
2082-
'X-Robots-Tag': 'none',
2082+
'X-Robots-Tag': 'noindex, nofollow',
20832083
'X-Frame-Options': 'SAMEORIGIN',
20842084
'X-Permitted-Cross-Domain-Policies': 'none',
20852085
'Referrer-Policy': 'no-referrer',
@@ -2099,7 +2099,7 @@ describe('OC.SetupChecks tests', function() {
20992099
'Strict-Transport-Security': 'max-age=99999999; preload; includeSubDomains',
21002100
'X-XSS-Protection': '1; mode=block',
21012101
'X-Content-Type-Options': 'nosniff',
2102-
'X-Robots-Tag': 'none',
2102+
'X-Robots-Tag': 'noindex, nofollow',
21032103
'X-Frame-Options': 'SAMEORIGIN',
21042104
'X-Permitted-Cross-Domain-Policies': 'none',
21052105
'Referrer-Policy': 'no-referrer',

‎lib/private/legacy/OC_Response.php‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -99,7 +99,7 @@ public static function addSecurityHeaders() {
9999
header('X-Content-Type-Options: nosniff'); // Disable sniffing the content type for IE
100100
header('X-Frame-Options: SAMEORIGIN'); // Disallow iFraming from other domains
101101
header('X-Permitted-Cross-Domain-Policies: none'); // https://www.adobe.com/devnet/adobe-media-server/articles/cross-domain-xml-for-streaming.html
102-
header('X-Robots-Tag: none'); // https://developers.google.com/webmasters/control-crawl-index/docs/robots_meta_tag
102+
header('X-Robots-Tag: noindex, nofollow'); // https://developers.google.com/webmasters/control-crawl-index/docs/robots_meta_tag
103103
header('X-XSS-Protection: 1; mode=block'); // Enforce browser based XSS filters
104104
}
105105
}

‎lib/public/AppFramework/Http/Response.php‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -257,7 +257,7 @@ public function getHeaders() {
257257

258258
$this->headers['Content-Security-Policy'] = $this->getContentSecurityPolicy()->buildPolicy();
259259
$this->headers['Feature-Policy'] = $this->getFeaturePolicy()->buildPolicy();
260-
$this->headers['X-Robots-Tag'] = 'none';
260+
$this->headers['X-Robots-Tag'] = 'noindex, nofollow';
261261

262262
if ($this->ETag) {
263263
$mergeWith['ETag'] = '"' . $this->ETag . '"';

‎tests/data/setUploadLimit/htaccess‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
# Add security and privacy related headers
1212
Header set X-Content-Type-Options "nosniff"
1313
Header set X-XSS-Protection "1; mode=block"
14-
Header set X-Robots-Tag "none"
14+
Header set X-Robots-Tag "noindex, nofollow"
1515
Header set X-Frame-Options "SAMEORIGIN"
1616
SetEnv modHeadersAvailable true
1717
</IfModule>

‎tests/lib/AppFramework/Controller/ControllerTest.php‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -116,7 +116,7 @@ public function testFormatDataResponseJSON() {
116116
'Content-Security-Policy' => "default-src 'none';base-uri 'none';manifest-src 'self';frame-ancestors 'none'",
117117
'Feature-Policy' => "autoplay 'none';camera 'none';fullscreen 'none';geolocation 'none';microphone 'none';payment 'none'",
118118
'X-Request-Id' => $this->request->getId(),
119-
'X-Robots-Tag' => 'none',
119+
'X-Robots-Tag' => 'noindex, nofollow',
120120
];
121121

122122
$response = $this->controller->customDataResponse(['hi']);

0 commit comments

Comments
 (0)