📄 025_mm_vmscan_c.html
字号:
} /* used to insert page numbers */ div.google_header::before, div.google_footer::before { position: absolute; top: 0; } div.google_footer { flow: static(footer); } /* always consider this element at the start of the doc */ div#google_footer { flow: static(footer, start); } span.google_pagenumber { content: counter(page); } span.google_pagecount { content: counter(pages); } } @page { @top { content: flow(header); } @bottom { content: flow(footer); } } /* end default print css */ /* custom css *//* end custom css */ /* ui edited css */ body { font-family: Verdana; font-size: 10.0pt; line-height: normal; background-color: #ffffff; } .documentBG { background-color: #ffffff; } /* end ui edited css */</style> </head> <body revision="dcbsxfpf_66dzt89wc3:10"> <div align=center>
<table align=center border=0 cellpadding=0 cellspacing=0 height=5716 width=768>
<tbody>
<tr>
<td height=5716 valign=top width=100%>
<pre>2006-8-11 <br> mm/vmscan.c <br> <br> <br> I)概述 <br> <br>在分析page alloc的时候,分析过linux对内存页面的空闲量有些什么样的要求:<br><br>1)可分配页面的保有量要求:inactive_clean+free pages(in buddy pages)<br> 系统的期望值是freepages.high + inactive_target / 3,inactive_target就是<br>min((memory_pressure >> INACTIVE_SHIFT),num_physpages / 4)).可见期望的保有量有动态的因素在内.因为memory_pressure<br>是一段时间内的平均值,根据内存需求的不同所期望的保有量也不一样.<br> <br> 而现在的保有量是nr_free_pages() + nr_inactive_clean_pages();<br> 函数free_shortage,计算期望的可分配页面和现实之差距.如果保有量合格,就看zone中的inbuddy free pages是否比期望值少.<br>只要有一个保有量不合格,就必须立即加以调整.<br><br>2)潜在可分配页面的保有量要求(inactive_shortage):<br> 潜在可分配页面就是:buddyfree+inactiveclean+inactive_dirty<br> 期望保有量:freepages.high+inactive_target<br> 现存量:<br> nr_free_pages()+nr_inactive_clean_pages()+nr_inactive_dirty_pages.<br> <br> 保证这两种页面有一个合理的水平,这样在分配内存的时候,(期望)一般不会出现内存紧缺.free_shortage,inactive_shortage这两<br>个函数不复杂,略.<br> <br> 这个文件就是处理lru,负责页面的回收. 包括映射断裂,逐级的调整页面所在的lru 队列,<br>直至最后回到buddy 系统. 另外对于dcache,icache, slab系统的空闲页面,在内存紧张的<br>时候也要进行回收.从而保证空闲内存保有量得到满足.<br><br>lru cache的组成,前面已经有叙述了:<br> struct list_head active_list;<br> struct list_head inactive_dirty_list;<br> 每个zone 中的inactive clean 队列.<br> <br> 为了完成这些任务,有两个内核线程,kswapd和kreclaimd,见下图. 图中忽略了调用的条件<br>简单的列出了每个线程的作用: <br><br>kswapd <br> |----do_try_to_free_pages <br> | page_launder (inactive_dirty--->zone clean) <br> shrink_dcache_memory <br> shrink_icache_memory <br> refill_inactive <br> kmem_cache_reap <br> refill_inactive_scan (active---->inactive) <br> shrink_dcache_memory <br> shrink_icache_memory <br> swap_out (prcess vss-> active list or inactive) <br> kmem_cache_reap (slab) <br> <br> +--> refill_inactive_scan (active---->inactive) <br> <br> +--> run_task_queue <br> <br> +--> oom_kill <br> <br> <br> <br> kreclaimd ( zone_t->inactive_clean_pages ----> zone_t buddy )<br> reclaim_page <br> _free_page <br> <br> <br> <br> 为了更加直观,请看下面的图:(page alloc中也有) <br> <br>mm->rss(进程页面) ************************************************************ <br> |swap_out->try_to_swap_out <br>active_list->*********************** ******** | <br> /\ |refill_inactive_scan | <br> |page_launder | | <br> | \ / \ / <br>inactive_dirty_list->***************************************************** <br> |page_launder (wite back to disk if necessary) <br> \ / <br>zone_t->inactive_clean_pages->*********************** (__alloc_pages: reclaim_page) <br> |kreclaimd->reclaim_page, __free_pages <br> \ / <br>zone_t->free_area(buddy)->*********************** <br><br><br> II)swap out<br> <br> 一次扫描一定量的进程页面,从不太忙碌的进程中回收可用页面是保证空闲页面保有量,<br>实现VM(back store)的一个重要操作.<br> swap_out选择一个进程(当然不包括内核),通过try_to_swap_out断开此进程对页面的映<br>射.根据页面的熟悉(是否已经在swap,是否是page cache(page->mapping)),采取不同操作.<br> 进程有一个rss统计,还有一个swap_cnt=(mm->rss >> SWAP_SHIFT),swap cnt代表的是<br>这个进程在swap out的过程中应该贡献的页面数量.(虽然不一定能找到合适页面)所以swap<br>cnt大的进程优先被选中.<br> <br>/*<br> * 选择swap_cnt 值最大的任务试着交换出一些空间.swap_cnt 最大表明他最应该受到检查.<br> * 看看能不能换出一些页面. (try_to_swap_out)<br> * N.B. This function returns only 0 or 1. Return values != 1 from<br> * the lower level routines result in continued processing.<br> */<br>#define SWAP_SHIFT 5<br>#define SWAP_MIN 8<br>static int swap_out(unsigned int priority, int gfp_mask)<br>{<br> int counter;<br> int __ret = 0;<br><br> /* <br> * 进行两种方式的扫描, assign = 0,按照swap_cnt选择进程.<br> * assign = 1,重新计算rss 和swap_cnt 的值,然后同时根据<br> * swp_cnt选择合适进程.<br> *<br> * 如果选中的进程不能提供空闲页面,则清楚swap_cnt(swap_out_mm),这样 <br> * 任务就不会再次被选中.<br> */<br> counter = (nr_threads << SWAP_SHIFT) >> priority;<br> if (counter < 1)<br> counter = 1;<br><br> for (; counter >= 0; counter--) { //这个循环次数受优秀级控制<br> struct list_head *p; //代表了努力程度<br> unsigned long max_cnt = 0;<br> struct mm_struct *best = NULL;<br> int assign = 0; /*assign 在某一轮尝试如果不能找到一个best,<br> 则进行第二种方式的扫描*/<br> int found_task = 0;<br> select:<br> spin_lock(&mmlist_lock);<br> p = init_mm.mmlist.next;<br> for (; p != &init_mm.mmlist; p = p->next) {<br> struct mm_struct *mm = list_entry(p, struct mm_struct, mmlist);<br> if (mm->rss <= 0)<br> continue;<br> found_task++;<br> /* Refresh swap_cnt? */<br> if (assign == 1) { //选择进程同时重新计算swap cnt<br> mm->swap_cnt = (mm->rss >> SWAP_SHIFT);<br> if (mm->swap_cnt < SWAP_MIN)<br> mm->swap_cnt = SWAP_MIN;<br> }<br> if (mm->swap_cnt > max_cnt) {<br> max_cnt = mm->swap_cnt;<br> best = mm;<br> }<br> }<br><br> /* Make sure it doesn't disappear */<br> if (best) <br> atomic_inc(&best->mm_users);<br> spin_unlock(&mmlist_lock);<br><br> /*<br> * We have dropped the tasklist_lock, but we<br> * know that "mm" still exists: we are running<br> * with the big kernel lock, and exit_mm()<br> * cannot race with us.<br> */<br> if (!best) { /* 找不到best 说明swap_cnt 都变成了0*/<br> if (!assign && found_task > 0) {<br> assign = 1;<br> goto select;<br> }<br> break;<br> } else {<br> __ret = swap_out_mm(best, gfp_mask);<br> mmput(best);<br> break;<br> }<br> }<br> return __ret;<br>}<br><br> 然后看看try_to_swap_out,其他swap out相关的函数不再分析.<br>/*<br> * rss(驻留页面集合)<br> *<br> * 希望caller 继续就返回0, 否则返回1.<br> *<br> * NOTE! If it sleeps, it *must* return 1 to make sure we<br> * don't continue with the swap-out. Otherwise we may be<br> * using a process that no longer actually exists (it might<br> * have died while we slept).<br> */<br>static int try_to_swap_out(struct mm_struct * mm, struct vm_area_struct* vma, <br> unsigned long address, pte_t * page_table, <br> int gfp_mask)<br>{<br><br><br> pte_t pte;<br> swp_entry_t entry;<br> struct page * page;<br> int onlist;<br><br> pte = *page_table;<br> if (!pte_present(pte))<br> goto out_failed;<br> page = pte_page(pte);<br> if ((!VALID_PAGE(page)) || PageReserved(page))<br> goto out_failed;<br><br> if (!mm->swap_cnt)<br> return 1;<br><br> mm->swap_cnt--; /*swap_cnt: 根据rss,有个初始值每检查一个页面减1 */<br> <br><br> //是否在activ list<br> onlist = PageActive(page);<br> <br> /* 如果最近被访问过,提升age */<br> if (ptep_test_and_clear_young(page_table)) {<br> age_page_up(page);<br> goto out_failed;<br> }<br><br><br> /*最近未被访问,则降低age*/<br> if (!onlist) /* 不在activ list age down 由swap_out负责, refill_inactive_scan <br> 负责active list 的age down*/<br> age_page_down_ageonly(page);<br><br> <br> /*寿命未尽,不管*/<br> if (page->age > 0)<br> goto out_failed;<br> <br><br> if (TryLockPage(page))<br> goto out_failed;<br><br><br> /* age 耗尽可以断开映射了*/<br> pte = ptep_get_and_clear(page_table);<br> flush_tlb_page(vma, address);<br><br><br> <br> if (PageSwapCache(page)) {<br> /* <br> * 页面已经在交换空间(swap cache)了,<br> * 返回0, 通知上层继续扫描.<br> */<br> entry.val = page->index;<br> if (pte_dirty(pte))<br> set_page_dirty(page);<br>set_swap_pte:<br> swap_duplicate(entry); /*此进程引用此swap entry*/<br> set_pte(page_table, swp_entry_to_pte(entry));<br>drop_pte:<br> UnlockPage(page);<br> mm->rss--;<br><br> <br> deactivate_page(page); /* active->inactive, if on list*/<br><br> page_cache_release(page); /*此进程不再能够使用此页面*/<br>out_failed:<br> return 0;<br> }<br><br> /*<br> * 下面判断一下这个page 是否是一个clean page.<br> * 如果是我们就可以断开映射了.<br> * <br> * 如果不是的话我们应该将page->flags <br> * 的dity标志置位.<br> */<br> flush_cache_page(vma, address);<br> if (!pte_dirty(pte))<br> goto drop_pte;<br><br> /*<br> * dity 页,看看是否属于一个mapping,上面的PageSwapCache<br> * 只是判断是否在swapper_space,只是mapping的一个特例.<br> */<br> if (page->mapping) {<br> set_page_dirty(page);<br> goto drop_pte;<br> }<br><br> /*<br> * dity, 还没有交换空间<br> * 分配一个交换空间给他<br> */<br> entry = get_swap_page();<br> if (!entry.val)<br> goto out_unlock_restore; /* No swap space left */<br><br> <br> /* 挂入swap cache, 因为page->age 为零, <br> * 不会挂入入全局active 队列.<br> * 同时置位flag 的dity 位.<br> */<br> add_to_swap_cache(page, entry);<br><br> set_page_dirty(page);<br><br> goto set_swap_pte;<br><br>out_unlock_restore:<br> set_pte(page_table, pte);<br> UnlockPage(page);<br> return 0;<br>}<br><br><br> swap out扫描的过程中根据页面最近是否被访问过,调整页面的age值,对于age=0的页面<br>可以断开其映射. 可以看到page cache(mapping)和swap cache的不同处理.<br> swap空间的页面,pte记录的是swap entry. 普通maping则直接断开映射. pte置0.<br> <br> <br> <br> III) page_launder<br> <br> 将dity 的页面清洗成clean 并移入clean 队列. 对inactive list 扫描两次,第一次把已<br>经干净的页面移入inactive_clean 链表,第二次异步回写dirty 页面.当kswapd 无法供应所需<br>的页面时,则进行同步的回写. 原作者已经写了很多注释.仔细看看吧.有问题则讨论.<br>#define MAX_LAUNDER (4 * (1 << page_cluster))<br>int page_launder(int gfp_mask, int sync)<br>{<br> int launder_loop, maxscan, cleaned_pages, maxlaunder;<br> int can_get_io_locks;<br> struct list_head * page_lru;<br> struct page * page;<br><br> /*<br> * We can only grab the IO locks (eg. for flushing dirty<br> * buffers to disk) if __GFP_IO is set.<br> */<br> can_get_io_locks = gfp_mask & __GFP_IO;<br><br> launder_loop = 0;<br> maxlaunder = 0;<br> cleaned_pages = 0;<br><br>dirty_page_rescan:<br> spin_lock(&pagemap_lru_lock);<br> maxscan = nr_inactive_dirty_pages;<br> while ((page_lru = inactive_dirty_list.prev) != &inactive_dirty_list &&<br> maxscan-- > 0) {<br> page = list_entry(page_lru, struct page, lru);<br><br> /* Wrong page on list?! (list corruption, should not happen) */<br> if (!PageInactiveDirty(page)) {<br> printk("VM: page_launder, wrong page on list.\n");<br> list_del(page_lru);<br> nr_inactive_dirty_pages--;<br> page->zone->inactive_dirty_pages--;<br> continue;<br> }<br><br> /* Page is or was in use? Move it to the active list. */<br> if (PageTestandClearReferenced(page) || page->age > 0 ||<br> (!page->buffers && page_count(page) > 1) ||<br> page_ramdisk(page)) {<br> del_page_from_inactive_dirty_list(page);<br> add_page_to_active_list(page);<br> continue;<br> }<br><br> /*<br> * The page is locked. IO in progress?<br> * Move it to the back of the list.<br> */<br> if (TryLockPage(page)) {<br> list_del(page_lru);<br> list_add(page_lru, &inactive_dirty_list);<br> continue;<br> }<br><br> /*<br> * Dirty swap-cache page? Write it out if<br> * last copy..<br> */<br> if (PageDirty(page)) {<br> int (*writepage)(struct page *) = page->mapping->a_ops->writepage;<br> int result;<br><br> if (!writepage)<br> goto page_active;<br><br> /* First time through? Move it to the back of the list */<br> if (!launder_loop) {<br> list_del(page_lru);<br> list_add(page_lru, &inactive_dirty_list);<br> UnlockPage(page);<br> continue;<br> }<br><br> /* OK, do a physical asynchronous write to swap. */<br> ClearPageDirty(page);<br> page_cache_get(page);<br> spin_unlock(&pagemap_lru_lock);<br><br> result = writepage(page);<br> page_cache_release(page);<br><br> /* And re-start the thing.. */<br> spin_lock(&pagemap_lru_lock);<br> if (result != 1)<br> continue;<br> /* writepage refused to do anything */<br> set_page_dirty(page);<br> goto page_active;<br> }<br><br> /*<br> * If the page has buffers, try to free the buffer mappings<br> * associated with this page. If we succeed we either free<br> * the page (in case it was a buffercache only page) or we<br> * move the page to the inactive_clean list.<br> *<br> * On the first round, we should free all previously cleaned<br> * buffer pages<br> */<br> if (page->buffers) {<br> int wait, clearedbuf;<br> int freed_page = 0;<br> /*<br> * Since we might be doing disk IO, we have to<br> * drop the spinlock and take an extra reference<br> * on the page so it doesn't go away from under us.<br> */<br> del_page_from_inactive_dirty_list(page);<br> page_cache_get(page);<br> spin_unlock(&pagemap_lru_lock);<br><br> /* Will we do (asynchronous) IO? */<br> if (launder_loop && maxlaunder == 0 && sync)<br> wait = 2; /* Synchrounous IO */<br> else if (launder_loop && maxlaunder-- > 0)<br> wait = 1; /* Async IO */<br> else<br> wait = 0; /* No IO */<br><br> /* Try to free the page buffers. */<br> clearedbuf = try_to_free_buffers(page, wait);<br><br> /*<br> * Re-take the spinlock. Note that we cannot<br> * unlock the page yet since we're still<br> * accessing the page_struct here...<br> */<br> spin_lock(&pagemap_lru_lock);<br><br> /* The buffers were not freed. */<br> if (!clearedbuf) {<br> add_page_to_inactive_dirty_list(page);<br><br> /* The page was only in the buffer cache. */<br> } else if (!page->mapping) {<br> atomic_dec(&buffermem_pages);<br> freed_page = 1;<br> cleaned_pages++;<br><br> /* The page has more users besides the cache and us. */<br> } else if (page_count(page) > 2) {<br> add_page_to_active_list(page);<br><br> /* OK, we "created" a freeable page. */<br> } else /* page->mapping && page_count(page) == 2 */ {<br> add_page_to_inactive_clean_list(page);<br> cleaned_pages++;<br> }<br><br> /*<br> * Unlock the page and drop the extra reference.<br> * We can only do it here because we ar accessing<br> * the page struct above.<br> */<br> UnlockPage(page);<br> page_cache_release(page);<br><br> /* <br> * If we're freeing buffer cache pages, stop when<br> * we've got enough free memory.<br> */<br> if (freed_page && !free_shortage())<br> break;<br> continue;<br> } else if (page->mapping && !PageDirty(page)) {<br> /*<br> * If a page had an extra reference in<br> * deactivate_page(), we will find it here.<br> * Now the page is really freeable, so we<br> * move it to the inactive_clean list.<br> */<br> del_page_from_inactive_dirty_list(page);<br> add_page_to_inactive_clean_list(page);<br> UnlockPage(page);<br> cleaned_pages++;<br> } else {<br>page_active:<br> /*<br> * OK, we don't know what to do with the page.<br> * It's no use keeping it here, so we move it to<br> * the active list.<br> */<br> del_page_from_inactive_dirty_list(page);<br> add_page_to_active_list(page);<br> UnlockPage(page);<br> }<br> }<br> spin_unlock(&pagemap_lru_lock);<br><br> /*<br> * If we don't have enough free pages, we loop back once<br> * to queue the dirty pages for writeout. When we were called<br> * by a user process (that /needs/ a free page) and we didn't<br> * free anything yet, we wait synchronously on the writeout of<br> * MAX_SYNC_LAUNDER pages.<br> *<br> * We also wake up bdflush, since bdflush should, under most<br> * loads, flush out the dirty pages before we have to wait on<br> * IO.<br> */<br> if (can_get_io_locks && !launder_loop && free_shortage()) {<br> launder_loop = 1;<br> /* If we cleaned pages, never do synchronous IO. */<br> if (cleaned_pages)<br> sync = 0;<br> /* We only do a few "out of order" flushes. */<br> maxlaunder = MAX_LAUNDER;<br> /* Kflushd takes care of the rest. */<br> wakeup_bdflush(0);<br> goto dirty_page_rescan;<br> }<br><br> /* Return the number of pages moved to the inactive_clean list. */<br> return cleaned_pages;<br>}<br><br><br><br> <br> <br> IV) refill_inactive_scan<br><br>/*<br> * refill_inactive_scan - 扫描active 列表找到应该deactivate 的页面<br> * @priority: the priority at which to scan<br> * @oneshot: exit after deactivating one page<br> *<br> * This function will scan a portion of the active list to find<br> * unused pages, those pages will then be moved to the inactive list.<br> */<br> /*<br> * 在acitve_list 中的页面有两种情况:<br> *<br> * 1. 拥有mapping 的page <br> * 2. 后来加入这个队列的(加入swapper mapping).age 为0，<br> * 但是有可能随时恢复映射 <br> */<br>int refill_inactive_scan(unsigned int priority, int oneshot)<br>{<br> struct list_head * page_lru;<br> struct page * page;<br> int maxscan, page_active = 0;<br> int ret = 0;<br><br> /* Take the lock while messing with the list... */<br> spin_lock(&pagemap_lru_lock);<br><br> maxscan = nr_active_pages >> priority; <br> //队尾的页面被换出的概率小<br> <br> while (maxscan-- > 0 && (page_lru = active_list.prev) != &active_list) {<br> page = list_entry(page_lru, struct page, lru);<br><br> /* Wrong page on list?! (list corruption, should not happen) */<br> if (!PageActive(page)) {<br> printk("VM: refill_inactive, wrong page on list.\n");<br> list_del(page_lru);<br> nr_active_pages--;<br> continue;<br> }<br><br> /* Do aging on the pages. */<br> if (PageTestandClearReferenced(page)) {<br> // 内核持有此页面(如touch_buffer)<br> age_page_up_nolock(page);<br> page_active = 1; <br> //置此标记会把page 移到队尾<br> } else {<br> age_page_down_ageonly(page);<br> /*<br> * Since we don't hold a reference on the page<br> * ourselves, we have to do our test a bit more<br> * strict then deactivate_page(). This is needed<br> * since otherwise the system could hang shuffling<br> * unfreeable pages from the active list to the<br> * inactive_dirty list and back again...<br> *<br> * SUBTLE: we can have buffer pages with count 1.<br> */<br> if (page->age == 0 && page_count(page) <=<br> (page->buffers ? 2 : 1)) {<br> deactivate_page_nolock(page); <br> page_active = 0;<br> } else {<br> page_active = 1;<br> }<br> }<br> /*<br> * If the page is still on the active list, move it<br> * to the other end of the list. Otherwise it was<br> * deactivated by age_page_down and we exit successfully.<br> */<br> if (page_active || PageActive(page)) { <br> // 把页面移到队尾<br> list_del(page_lru);<br> list_add(page_lru, &active_list); <br> } else {<br> ret = 1;<br> if (oneshot)<br> break;<br> }<br> }<br> spin_unlock(&pagemap_lru_lock);<br><br> return ret;<br>}<br><br> 再次讨论PageTestandClearReferenced.看看2.6关于此位的说明:<br> * For choosing which pages to swap out, inode pages carry a PG_referenced bit,<br> * which is set any time the system accesses that page through the (mapping,<br> * index) hash table. This referenced bit, together with the referenced bit<br> * in the page tables, is used to manipulate page->age and move the page across<br> * the active, inactive_dirty and inactive_clean lists.<br> <br> 意思是,对于缓存文件内容的页面,除了来自用户进程的访问,内核本身访问此页面也应该<br>age up.<br> 对于2.4内核,这种页面主要来自buffer,见grow_buffers,getblk. buffer中的页面也使用<br>lru队列进行swap.但是有可能没有映射到用户页面(blk dev 文件的读取),无法age up.只好<br>通过touch_buffer进行by hand的age up.<br><br> <br> page->count不死<br> <br> 关于 page_launder 还有一个话题,就是这个函数标号page_active的地方.论坛中有很多关<br>于这里的讨论.仅发表一下看法,供大家参考.<br> 以前的讨论中提到page_active:处的条件中隐含有<br> (page->count==1&&page->mapping==0&&page->buffer==0)<br> 其实这个标号的条件是:<br> 1)no Buffer, no mapping(one process)(dirty can't write out or not), <br> 2)no buffer, mapping,dirty but can't write out)*/<br> <br> 对于第一种页面,在2.4.0的内核中,似乎就没有可能进入lru cache.(2.4.20有). 因为查看<br>页面所有进入lru的途径,必然是,或者有mapping,或者有buffer. 并且进入lru后,没有找到恢<br>复进程映射时将mapping,buffer去掉而留在lru中的情况.<br> 关于lru, page_launder,linus有一个讨论可以参考一下: <br><a href=http://groups.google.com/group/fa.linux.kernel/browse_thread/thread/2e175262f3af7dc3/7dad9a48a4cf9fc2?q=page_launder+page_active%3A&lnk=ol&hl=zh-CN& id=d6h3 title="page_launder, linux option">page_launder, linux option</a> <br> linus认为,进入inactive dirty的page,不应该没有mapping的.否则dirty了又能如何.当然<br>2.4.20以后这种页面可以存在的,但是swap out还是会给他分配mapping(swap space),只是临<br>存在.<br> 第二种倒是有,比如ramfs,以前的tmpfs也是.<br> <br> 本论坛还有一个对这个函数的讨论,值得一看:<br><a href=http://www.linuxforum.net/forum/showthreaded.php?Cat=&Board=linuxK&Number=249364&page=&view=&sb=&o= id=p-bv title=linuxfourm关于这个问题的讨论>linuxfourm关于这个问题的讨论</a> <br> 以上的分析也希望对这个问题有所帮助. <br> <br> <br> 其他函数不再讨论.swap out的过程通过各种手段进行调整,经过不断的演化,成了现在这<br>个样子.我感觉应该把握的是这种思路,然后看代码的时候就不觉得很乱了.<br><br> linux mm分析暂告一个段落.<br> <br> <br> <br> 2006.8.11<br></pre>
</td>
</tr>
</tbody>
</table>
</div></body></html>⌨️ 快捷键说明
复制代码
Ctrl + C
搜索代码
Ctrl + F
全屏模式
F11
切换主题
Ctrl + Shift + D
显示快捷键
?
增大字号
Ctrl + =
减小字号
Ctrl + -