问题
I have been developing a Joomla site with version 2.5.11.Site will be under very high traffic.
My problem is about MySQL query performance. Database includes about 60000 rows in content table, and the query seen below (core com_content articles model query) execution time is about 6 seconds.Very slow.
SELECT a.id,
a.title,
a.alias,
a.title_alias,
a.introtext,
a.checked_out,
a.checked_out_time,
a.catid,
a.created,
a.created_by,
a.created_by_alias,
CASE
WHEN a.modified = 0 THEN a.created
ELSE a.modified
END as modified,
a.modified_by,
uam.name as modified_by_name,
CASE
WHEN a.publish_up = 0 THEN a.created
ELSE a.publish_up
END as publish_up,
a.publish_down,
a.images,
a.urls,
a.attribs,
a.metadata,
a.metakey,
a.metadesc,
a.access,
a.hits,
a.xreference,
a.featured,
LENGTH(a.fulltext) AS readmore,
CASE
WHEN badcats.id is not null THEN 0
ELSE a.state
END AS state,
c.title AS category_title,
c.path AS category_route,
c.access AS category_access,
c.alias AS category_alias,
CASE
WHEN a.created_by_alias > ' ' THEN a.created_by_alias
ELSE ua.name
END AS author,
ua.email AS author_email,
contact.id as contactid,
parent.title as parent_title,
parent.id as parent_id,
parent.path as parent_route,
parent.alias as parent_alias,
ROUND(v.rating_sum / v.rating_count, 0) AS rating,
v.rating_count as rating_count,
c.published,
CASE
WHEN badcats.id is null THEN c.published
ELSE 0
END AS parents_published
FROM #__content AS a
LEFT JOIN #__content_frontpage AS fp ON fp.content_id = a.id
LEFT JOIN #__categories AS c ON c.id = a.catid
LEFT JOIN #__users AS ua ON ua.id = a.created_by
LEFT JOIN #__users AS uam ON uam.id = a.modified_by
LEFT JOIN
(SELECT contact.user_id, MAX(contact.id) AS id, contact.language FROM
#__contact_details AS contact WHERE contact.published = 1 GROUP BY
contact.user_id, contact.language) AS contact ON contact.user_id =
a.created_by
LEFT JOIN #__categories as parent ON parent.id = c.parent_id
LEFT JOIN #__content_rating AS v ON a.id = v.content_id
LEFT OUTER JOIN
(SELECT cat.id as id FROM #__categories AS cat JOIN #__categories AS
parent ON cat.lft BETWEEN parent.lft AND parent.rgt WHERE parent.extension
= 'com_content' AND parent.published != 1 GROUP BY cat.id) AS badcats ON
badcats.id = c.id
WHERE CASE
WHEN badcats.id is null THEN a.state
ELSE 0
END = 1 AND
a.featured = 0 AND
a.id NOT IN (8921, 33722, 33728, 33729, 34187, 35047, 36784, 36236, 33724,
19522) AND
a.catid IN (8, 39, 40, 38, 72, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19,
20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 36, 37, 41) AND
(a.publish_up = '0000-00-00 00:00:00' OR
a.publish_up <= '2013-06-12 06:44:44') AND
(a.publish_down = '0000-00-00 00:00:00' OR
a.publish_down >= '2013-06-12 06:44:44')
GROUP BY a.id,
a.title,
a.alias,
a.title_alias,
a.introtext,
a.checked_out,
a.checked_out_time,
a.catid,
a.created,
a.created_by,
a.created_by_alias,
a.created,
a.modified,
a.modified_by,
uam.name,
a.publish_up,
a.attribs,
a.metadata,
a.metakey,
a.metadesc,
a.access,
a.hits,
a.xreference,
a.featured,
a.fulltext,
a.state,
a.publish_down,
badcats.id,
c.title,
c.path,
c.access,
c.alias,
uam.id,
ua.name,
ua.email,
contact.id,
parent.title,
parent.id,
parent.path,
parent.alias,
v.rating_sum,
v.rating_count,
c.published,
c.lft,
a.ordering,
parent.lft,
fp.ordering,
c.id,
a.images,
a.urls
ORDER BY publish_up DESC
LIMIT 4, 4
For example when I changed
(a.publish_up = '0000-00-00 00:00:00' OR
a.publish_up <= '2013-06-12 06:44:44') AND
(a.publish_down = '0000-00-00 00:00:00' OR
a.publish_down >= '2013-06-12 06:44:44')
with
(a.publish_up >= DATE_SUB(NOW(), INTERVAL 2 MONTH))
Query execution time is about ~0.5second
Query explain :
What can I do without hacking the core code,I never want to hack the core code...
MySQL server Config:
skip-external-locking
skip-name-resolve
key_buffer_size = 128M
max_connections=1024
max_allowed_packet = 16M
net_buffer_length = 8K
table_open_cache = 512
table_cache = 2048
join_buffer_size = 128M
read_buffer_size = 128M
sort_buffer_size = 128M
read_rnd_buffer_size=2M
myisam_sort_buffer_size=128M
thread_concurrency=16
query_cache_size = 128M
query_cache_type=1
thread_cache_size = 1300
query_cache_limit=128M
max_heap_table_size = 1024M
tmp_table_size = 1024M
回答1:
The best way I've found to improve that very slow query it to remove the references to archive (which defeat the entire purpose of the archive status which is to remove those items from list results). Just take items that are published (or published + unpublished) so eiher state=1 or state IN(0,1) You could look at possibly using http://extensions.joomla.org/extensions/style-a-design/templating/15611
Switching to InnoDB will possibly help also but having the most recent possible version of MySQL is the biggest way to get improvements.
About "what can you do now" I would love to see a pull request from you to add date limitations to that query. We have recently added data math to JDatabaseQuery so i is relatively simple to do and would be a really nice improvement. There are also some other pending issues that have big performance impacts and if you test and comment on them they will move forward into the code base for the next release. For example: https://github.com/joomla/joomla-cms/pull/1274
回答2:
Do you really have to do a GROUP BY on all those columns? Would a simple GROUP BY a.id suffice?
If that doesn't have an effect, try increasing the values of the variables tmp_table_size and max_heap_table_size. This might be why you're getting a "Using Temporary" in the "Extra column". This message is because MySQL ran out of physical memory while doing the sort and needs to write the data to disk first, which is slower.
Find out what these values are:SELECT @@tmp_table_size, @@max_heap_table_size;
And then set them to something higher, until the "Using Temporary" disappears. (hopefully you have enough memory to work with):SET tmp_table_size = ###, @@max_heap_table_size = ###;
There's more information on this page: http://dev.mysql.com/doc/refman/5.1/en/internal-temporary-tables.html
If the above doesn't work, you can try throwing each of your derived queries into a temporary table with indexes on each of the columns you're joining on, and join on each of the new temp tables. But this can make it slower, especially if you have to do it during each request. Otherwise if the data isn't updated, then it could work.
P.S. I just realized your comment "What can I do without hacking the core code". Hopefully the session variables work.
回答3:
Since you'd have to hack the core to make changes to the query, your best bet would be to add a new model/view to com_content. The addition would not be overwritten by updates to the core.
回答4:
i have the same big problem than the thread-opener The time for me first connect with site "Get www.HDsports.at" is more than 1,5 seconds. I thinks, the reason is that i have a network-transfer with 4 GiB per hour.
My hoster sends my a mysqlslow.log and i see lot of the queries what the thread-opener had:
# Time: 150408 19:35:15
# User@Host: dbo376405720[dbo376405720] @ localhost []
# Query_time: 2.358831 Lock_time: 0.000273 Rows_sent: 5293 Rows_examined: 174411
SET timestamp=1428514515;
SELECT a.id, a.title, a.alias, a.introtext, a.fulltext, a.checked_out, a.checked_out_time, a.catid, a.created, a.created_by, a.created_by_alias, CASE WHEN a.modified = '0000-00-00 00:00:00' THEN a.created ELSE a.modified END as modified, a.modified_by, uam.name as modified_by_name,CASE WHEN a.publish_up = '0000-00-00 00:00:00' THEN a.created ELSE a.publish_up END as publish_up,a.publish_down, a.images, a.urls, a.attribs, a.metadata, a.metakey, a.metadesc, a.access, a.hits, a.xreference, a.featured, a.language, LENGTH(a.fulltext) AS readmore,CASE WHEN badcats.id is not null THEN 0 ELSE a.state END AS state,c.title AS category_title, c.path AS category_route, c.access AS category_access, c.alias AS category_alias,CASE WHEN a.created_by_alias > ' ' THEN a.created_by_alias ELSE ua.name END AS author,ua.email AS author_email,parent.title as parent_title, parent.id as parent_id, parent.path as parent_route, parent.alias as parent_alias,ROUND(v.rating_sum / v.rating_count, 0) AS rating, v.rating_count as rating_count,c.published, CASE WHEN badcats.id is null THEN c.published ELSE 0 END AS parents_published
FROM ngie_content AS a
LEFT JOIN ngie_content_frontpage AS fp ON fp.content_id = a.id
LEFT JOIN ngie_categories AS c ON c.id = a.catid
LEFT JOIN ngie_users AS ua ON ua.id = a.created_by
LEFT JOIN ngie_users AS uam ON uam.id = a.modified_by
LEFT JOIN ngie_categories as parent ON parent.id = c.parent_id
LEFT JOIN ngie_content_rating AS v ON a.id = v.content_id
LEFT OUTER JOIN (SELECT cat.id as id FROM ngie_categories AS cat JOIN ngie_categories AS parent ON cat.lft BETWEEN parent.lft AND parent.rgt WHERE parent.extension = 'com_content' AND parent.published != 1 GROUP BY cat.id ) AS badcats ON badcats.id = c.id
WHERE a.access IN (1,1,5) AND c.access IN (1,1,5) AND CASE WHEN badcats.id is null THEN a.state ELSE 0 END = 1 AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2015-04-08 17:35:12') AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2015-04-08 17:35:12')
ORDER BY c.lft, CASE WHEN a.publish_up = '0000-00-00 00:00:00' THEN a.created ELSE a.publish_up END DESC , a.created;
Here you can download the full file: http://forum.joomla.org/download/file.php?id=111296
i hope anyone can help me.
Best regards
来源：https://stackoverflow.com/questions/17065038/joomla-mysql-performance