• <tr id="yyy80"></tr>
  • <sup id="yyy80"></sup>
  • <tfoot id="yyy80"><noscript id="yyy80"></noscript></tfoot>
  • 99热精品在线国产_美女午夜性视频免费_国产精品国产高清国产av_av欧美777_自拍偷自拍亚洲精品老妇_亚洲熟女精品中文字幕_www日本黄色视频网_国产精品野战在线观看 ?

    Perceptual Image Outpainting Assisted by Low-Level Feature Fusion and Multi-Patch Discriminator

    2022-08-23 02:18:02XiaojieLiYongpengRenHongpingRenCanghongShiXianZhangLutaoWangImranMumtazandXiWu
    Computers Materials&Continua 2022年6期

    Xiaojie Li,Yongpeng Ren,Hongping Ren,Canghong Shi,Xian Zhang,Lutao Wang,Imran Mumtaz and Xi Wu,

    1College of Computer Science,Chengdu University of Information Technology,Chengdu,610225,China

    2Xihua University,Chengdu,610039,China

    3University of Agriculture Faisalabad,Pakistan

    Abstract: Recently,deep learning-based image outpainting has made greatly notable improvements in computer vision field.However, due to the lack of fully extracting image information, the existing methods often generate unnatural and blurry outpainting results in most cases.To solve this issue,we propose a perceptual image outpainting method, which effectively takes the advantage of low-level feature fusion and multi-patch discriminator.Specifically, we first fuse the texture information in the low-level feature map of encoder,and simultaneously incorporate these aggregated features reusability with semantic (or structural) information of deep feature map such that we could utilize more sophisticated texture information to generate more authentic outpainting images.Then we also introduce a multi-patch discriminator to enhance the generated texture, which effectively judges the generated image from the different level features and concurrently impels our network to produce more natural and clearer outpainting results.Moreover, we further introduce perceptual loss and style loss to effectively improve the texture and style of outpainting images.Compared with the existing methods,our method could produce finer outpainting results.Experimental results on Places2 and Paris StreetView datasets illustrated the effectiveness of our method for image outpainting.

    Keywords: Deep learning; image outpainting; low-level feature fusion;multi-patch discriminator

    1 Introduction

    Nowadays,artificial intelligence(AI)has ushered in a new big data era.The improvement of AI also promotes deep learning technology to be widely employed in many fields[1–4],especially in image processing field.Combining deep learning technology with image processing method,AI system could acquire more available environmental information to make correct decisions.For example,applying deep learning-based image processing to the pattern recognition and automatic control field for the efficient analysis and real-time response,which is considered as a promising prospect.

    Recently, deep learning-based methods [5–8] have been widely applied to image inpainting task and have made remarkable achievements.Image inpainting,as a common image editing task,aims to restore damaged images and remove objects.Existing image inpainting methods can mainly be divided into two groups:non-learning methods and learning-based methods.The former group is composed of diffusion-based [9,10] and distribution-based approaches [11,12].Concretely, the diffusion-based approaches use the texture synthesis to fill the unknown parts, and search or collect the suitable pixels of known regions to diffuse into the unknown regions.These methods can generate meaningful textures for the missing regions.However, they generate inpainting results often with blurry and distorted contents when meet a big hole or sophisticated textures, because they fail to capture the semantic information of images.On the other hand, the distribution-based approaches utilize the whole dataset to obtain the data distribution information, and finally generate inpainting images.Similarly, due to only extracting low-level pixel information, they can’t produce a fine texture.By contrary,learning methods[13–15]generally use convolutional neural networks to extract the semantic information of images such that they could realize a natural,realistic and plausible inpainting result.

    Compared with image inpainting, image outpainting is studied relatively fewer in the image processing field.It uses the known parts of images to recursively extrapolate a complete picture.Moreover, image outpainting faces a greater challenge because of the less neighboring pixel information.Furthermore,the outpainting model must produce plausible contents and vivid textures for the missing regions.In practice,image outpainting can be applied in panorama synthesis,texture synthesis and so on.The generative adversarial network(GAN)[16,17]is commonly employed in image outpainting,and it is suitable for unsupervised learning on complicated distribution.GAN,as a generative model,aims to train jointly its generator and discriminator for an adversarial idea.Specifically,the generator minimizes the loss function,and the discriminator maximizes the loss function.Since the adversarial training promotes the generator to capture the real data distribution, the network can generate fine and reasonable images.

    Existing image outpainting methods generally fail to effectively extract image information(such as structure and texture information), resulting in the unclarity and unnaturalness of outpainting results.To generate more semantically reasonable and visually natural outpainting results,we present a perceptual image outpainting method assisted by low-level feature fusion and multi-patch discriminator(LM).It is known that the low-level features map with higher resolution could acquire plentiful detail information(such as location information and texture information).However,it contains less semantic information.The high-level feature map could acquire more semantic information, and it perceives the less detail information.Therefore, we first fuse the texture information in the lowlevel feature map of encoder, and simultaneously incorporate these aggregated features reusability with semantic (structural) information of deep feature map by element-wise adding such that we could utilize more sophisticated texture information to generate more authentic outpainting images.Moreover, we introduce a multi-patch discriminator to enhance the generated texture information and comprehensively judge the reality of outpainting images.We design its outputs as an×ntensor equal to judge the number of patches of an image,which could perceive the relatively bigger receptive field.Therefore,our multi-patch discriminator further effectively judges the generated image from the different level features and indirectly promotes the generator to grasp the real distribution of input data.This could impel our network to produce more natural and clearer outpainting images.

    Furthermore,we employ perceptual loss[18]to extract the high-level feature information of both generated images and ground truths.Therefore, our network could restrain the texture generation of outpainting regions.Meanwhile, style loss [19] is employed to estimate the relevance of different features extracted by pre-trained Visual Geometry Group 19(VGG19)network[20],and we further compute a Gram matrix to obtain the global style of outpainting images.In this way,our model can generate real and consistent outpainting results.

    In general,our contributions are as follows:

    (1) We effectively fuse and reuse the texture information of the low-level feature map of encoder and simultaneously incorporate these aggregated features reusability with semantic(structural)information of deep feature map in the decoder,which could utilize more sophisticated texture information to generate more authentic outpainting results with finer texture.

    (2) We propose two multi-patch discriminators to comprehensively judge the generated images from the different level features,which further enlarges receptive field of discriminator network and finally improves the clarity and naturalness of outpainting results.

    The rest part of paper is organized as follows: Section 2 presents related image outpainting works.The detail theory of our proposed method is illustrated in Section 3.Section 4 introduces our experimental results which include qualitative and quantitative comparisons with existing methods.In the last section,we present conclusions and future works.

    2 Related Work

    In the early time,image inpainting fills the missing areas through non-learning methods,including patch-based [21–23] and diffusion-based methods [24–26].Caspi et al.[27] use bidirectional spatial similarity to maintain the information of input data, which can be applied in retargeting or image inpainting.Nonetheless,the spatial similarity estimation costs a large number of computation resources.Barnes et al.[28]propose a PatchMatch method,using a fast nearest neighbor estimation to match reasonable patches.Therefore, PatchMatch could save expensive computation cost.These methods all assume that the missing contents come from the known regions,thus they search and copy the patches of known areas to fill unknown areas.By this way,they can generally produce meaningful contents for the missing regions.However,they often exhibit badly for complicated structures or bigger holes,due to they only gain low-level image information such as the non-learning statistics information and simple pixel information of images.

    Context Encoder (CE) [29] firstly applies the deep learning-based and GAN-based method to image inpainting task.It presents a new unsupervised learning method which is based on contextual pixel prediction.CE can be used to generate realistic contents according to known pixel information.Its overall network is an encoder-decoder architecture.The encoder maps the missing image into the latent space,and then the decoder utilizes these features of latent space to generate missing contents.A channel-wise fully-connected layer is introduced to connected encoder and decoder.In addition,both reconstruction loss and adversarial loss are used to train the CE model for realizing a sharp inpainting result.In this way, CE could simultaneously obtain both structure representation and semantic information of images.However,owing to the limitation of the fully-connected layer in the network,it fails to produce clear inpainting results.

    Chao et al.[30] propose a multi-scale neural patch synthesis algorithm, which is composed of content network and texture network.It can generate fine content and texture through training jointly the two networks.The content network is used to fill contents for the missing areas, while the texture network is used to further improve texture of output results generated by content network.Furthermore, in the texture network, a pre-trained VGG network is employed to force the patches in the inpainting regions to be perceptually similar to the patches in the known regions.Since they fully take the texture of missing regions into account, the network performs well for producing fine structures.However,due to the multi-scale learning which costs a lot of computation resources,this method has significant limitations.

    Then,Iizuka et al.[31]present a novel image inpainting method which guarantees the inpainting images with both local and global consistency.More specifically, it uses a local discriminator and a global discriminator to realize fine inpainting results.The local discriminator judges the inpainting areas to achieve local detail consistency, while the global discriminator judges the whole image to ensure the consistent overall structure.Thanks to ensuring the consistency of local and global details,the model could produce much finer inpainting results.Moreover, it also achieves a more flexible inpainting without the limitation of image resolution and missing shape.

    To get over the influence of subordinate pixels in the missing regions,Liu et al.[32]create a partial convolution for irregular image inpainting.In the method, they use the masked and renormalized convolution to force the network to focus on the valid pixels of input images.Moreover, they also present a method to automatically update the mask value for the next convolutional layer.By this way,the influence of subordinate information can be reduced in some degree,which promotes the network to process the input image more effectively.Ultimately,they realize natural and clear inpainting results.

    Zheng et al.[33] propose a pluralistic image inpainting method (PICnet), which could produce multiple output for one input image.The most image inpainting methods only output one result,due to the limitation of one instance label provided by the ground truth.To let the model output diverse inpainting results, they invent a novel probabilistic theory to settle the problem.In addition, their network architecture contains two parallel paths,which are composed of the reconstructive path and the generated path.Concretely,the reconstructive path is used to obtain the distribution information of missing regions, and finally reconstructing a complete image.On the other hand, the generated path utilizes the distribution information of reconstructive path to guide the generation of missing images.By sampling from the variational auto-encoder(VAE)(another generative model),the network can produce pluralistic inpainting images.Owing to the considering of prior distribution of missing regions,they not only generate high-quality results but also create the diversity of images.

    Mark et al.[34] recently apply GAN to the image outpainting for painting outside the box(IOGnet).They employ the deep learning-based GAN approach to outpaint the panorama contents for the sides of missing images,and finally recursively expand the parts beyond the border.Furthermore,they adopt a three-stage strategy to stabilize the training process.In the first stage,the generator is trained by the L2 distance between the generated images and the ground truths.In the second stage,the discriminator is trained alone according to the adversarial loss.In the last stage,the generator and discriminator are trained jointly through the adversarial loss.Finally,the model could even generate a five-time outpainting result than the original input.However, the obscure contents appear in the outpainting parts.As a result,the work needs to be improved in some aspects.

    3 Perceptual Image Outpainting Assisted by Low-Level Feature Fusion and Multi-Patch Discriminator(LM)

    To produce high-quality outpainting results, we present a simple perceptual image outpainting method assisted by low-level feature fusion and multi-patch discriminator.Moreover, we simultaneously employ both perceptual loss and style loss to improve the texture and style of outpainting images.Network architecture will be introduced in Subsection 3.1, and the rest of subsections are used to introduce the principle of our method.

    3.1 Network Architecture

    As shown in Fig.1, a simple GAN-based network, mainly consisting of the generator and discriminator, is used in our network.Firstly, our encoder in generator maps input images (bothImandIc) into a latent feature space.We first fuse the texture information in the low-level feature map of encoder, and simultaneously incorporate these aggregated features reusability with semantic(structural) information of deep feature map by element-wise adding in decoder.This could utilize more sophisticated texture information to generate more authentic outpainting images.Furthermore,the inference module(yellow block)connects encoder with decoder for utilizing the latent feature more effectively.In fact,the inference module is equal to the function of VAE[35],which computes the mean and variance of latent features to sample useful features.Finally,to generate more realistic results,we inject outpainting image into the pre-trained VGG[36]network for obtaining the feature information,which will be used to compute perceptual loss and style loss.In addition, we use the Least Squares Generative Adversarial Network(LSGAN)loss[37]to stabilize the training of our model.Then we present a multi-patch discriminator to enhance the generated texture information, which effectively judges the generated image from the different level features and impels our network to produce more natural and clearer outpainting images.

    Figure 1:Overview of our network architecture

    3.1.1 Generator

    Fig.1 shows that our network structure consists of two paths:yellow path in the top and blue path in the bottom.Note that the former path aims to reconstruct inpainting images and the latter path aims to generate outpainting results.In the training, both masked imagesImandIc(complement ofIm)are concatenated by the channel-wise operation such that both can be simultaneously processed.Then we detach output features into both different inference modules(yellow block)to compute their latent features’mean and variance,which will be used to sample latent features.To simultaneously deal with both latent features,we concatenate both sampling features and feed them into the decoder.To easily grasp more sophisticated texture information and generate more authentic outpainting images,when decoder processes the latent features we fuse the texture information in the low-level feature map of encoder, and simultaneously incorporate these aggregated features reusability with semantic(structural) information of deep feature map by element-wise adding in decoder.It is formally defined as:

    whereFiis i-th layer’s aggregated features,Idenotes input image,Eiis i-th layer in the encoder, ⊕denotes channel-wise concatenation, andCDis down-sampling operation.Namely, we first downsample (i-1)-th layer’s features, and concatenate the down-sampling features with the i-th layer’s features by channel-wise concatenation.Therefore,Ficontains (i-1)-th and i-th layers’ aggregated feature information (see Eq.(1)).Then, we could pass aggregated featuresFiinto the decoder via element-wise adding.Therefore, the network could generate more sophisticated texture for the generated images.Finally,we produce both reconstructive imageIrecand generated imageIgen.

    3.1.2 Discriminator

    We design multi-patch discriminators (both Discriminator 1 and Discriminator 2) to enhance the generated texture information, which effectively judges the generated imageIgenandIrecfrom the different level features and impels our network to produce more natural and clearer outpainting images.Formally,it is defined as:

    whereis the generator’s adversarial loss,Diis the i-th layer of discriminator,andIgenis the generated image.Specifically, we judge the output patches in the last three layers of discriminator are real or fake.From the multi-patch information, the discriminator could effectively reinforce the ability of judgement for the output patch of discriminator (see Eq.(2)).Therefore, the discriminator could comprehensively judge an input image is real or fake.Finally,the real distribution of data is grasped by the generator,and the model could produce finer outpainting results.

    3.2 Perceptual Loss and Style Loss

    To further improve the texture and style of outpainting images and generate more realistic result,we simultaneously introduce both perceptual loss and style loss.Perceptual loss aims to extract semantic(structure)feature information via the pre-trained VGG19 network.By constraining theL1distance of these features,it can force outpainting results perceptually close to ground truths.Formally,the perceptual loss is defined as:

    whereIgenis the generated image andIgtis the ground truth.Φi(·)denotes the i-th layer features map of VGG.Actually, the perceptual loss is used to measure the difference of corresponding features extracted by VGG.The features in the convolutional neural network generally represent the semantic information of images such as the low-level textures or high-level attributes.Through penalizing these features dissimilar to the feature labels in the VGG, the outpainting parts can be improved in some degree.Thanks to the applying of perceptual loss in the training of GANs, the generator could be gradually tuned to produce a finer output result.

    Style loss aims to extract the general style of generated images and ground truth.Concretely,to capture the overall style,we calculate the Gram matrix of their features extracted by VGG network.As a result of theL1norm constraint on the corresponding Gram matrices, the outpainting images will approach the realistic style by degrees.Analogously,the style loss is defined as follow:

    where GΦi(·)denotes the Gram matrix of i-th layer’s feature extracted by VGG network.In fact,Gram matrix is the covariance matrix of eigenvectors in the Euclidean space,and it estimates the correlation of pair eigenvectors.Convolutional Neural Network(CNN)extracts the low-level texture information of images in the shallow layer,while in the deeper layer it obtains the high-level semantic information.The genuine attribute of an image is up to the combination of low-level and high-level information.Therefore, it can be used to measure the correlation of different features, including the important essence of images.Since we force the style of outpainting images to be similar to the style of ground truths,our model can produce the outpainting results with natural and authentic appearance.

    3.3 Other Loss

    Moreover,we apply the loss from PIC.Formally,

    where the subscriptrdenotes the reconstructive path (see yellow path in Fig.1), andgdenotes the generated path (see blue path in Fig.1).LKLis the KL loss for restraining the distribution of both reconstructive images and generated images.Lappis the reconstruction loss,andLadis the adversarial loss for GAN.

    In our model,the total loss is defined as follow:

    whereλ1=0.1,λ2=250.0 in our experiments.

    4 Experimental Results

    4.1 Dataset

    We evaluate our method on both Places2[38]and Paris StreetView[39]datasets.Places2 dataset is a natural scene dataset which is widely used in image outpainting.We divided Places2 into training set 308,500 and test set 20,000.Paris dataset is building view dataset,and we divided Paris into training set 14,900 and test set 100.All of images are resized to 128 × 128 and normalized to [0,1].These normalized inputs of[0,1]can accelerate the training of model,and it also summarizes the statistical distribution of uniform samples.

    4.2 Experimental Setup

    All experiments are implemented on Pytorch framework with Ubuntu 16.04, Python 3.6.9,PyTorch 1.2.0,and RTX 2080TI GPU.Moreover,we set a batch size of 64,and use Adam optimizer to train our network with an initial learning-rate of 0.00001, and the orthogonal method is used to initialize the parameters of model.Although the network consists of two paths,it is trained in an endto-end style.We also employ a LSGAN loss to make the training stable.In the training procedure,we update the discriminator once and update the generator once to complete the adversarial training.The test input is the masked image with missing center regular holes or long strips.Note that,during test,we only use the bottom blue path to output final results.During training time,our model spent 6 days and 5 days on Places2 and Paris datasets respectively, while PICnet spent 7 days and 6 days on Places2 and Paris datasets respectively.Therefore,it proves that our method is more efficient for training times.

    4.3 Evaluation Metrics

    We compare our method(PICnet-SP-LM)with PICnet and its variants(PICnet-S(PICnet with style loss) and PICnet-SP (PICnet with style loss and perceptual loss)) in terms of qualitative and quantitative aspects.In the qualitative aspect,we can visually judge whether the outpainting parts are fine or bad.In the quantitative aspect, six types of metrics are used to measure the performance of different methods:

    (1) Inception Score(IS)[40]is a common quantitative metric which is used to judge the quality of generated images.GANs,which can generate clear and diverse images,are considered as good generated models.IScan be used to measure the clarity and diversity of images.Formally,ISis defined as follow:

    wheregis the generator,ydenotes the generated image,andzis the label predicted by the pre-trained Inception V3 model.The higher IS score signifies that the generated images are clearer and more diverse.

    (2) Another metric usually used to measure the quality of GAN is Frechet Inception Distance(FID)[41].FID aims to estimate the distance between the feature vectors of generated image and ground truth in a same domain.Formally:

    wherexdenotes the ground truth andydenotes the generated image.μis the mean value of eigenvectors, andΣis the covariance matrix of eigenvectors.The lower FID score also means that the generated images are higher-quality for clarity and diversity.

    (3) Structural similarity (SSIM) aims to evaluate the quality of image based on the luminance,contract and structure of two images.Formally,

    wherex,ydenote ground truth and generated image respectively,μxis the mean value ofx,σxdenotes the variance ofx,andσxydenotes the covariance ofxandy.The higher SSIM means the generated images possess finer luminance,contract and structure.

    (4) Peak signal-to-noise ratio(PSNR)is a full reference estimation metric,and it is used to measure the degree of image distortion.Formally,

    wheredenotes the max pixel value in an image,andMSEis the abbreviation of mean square error.A higher PSNR score signifies the generated images are more natural.

    (5)L1loss measures the pixel-wise difference by computing the L1 distance.Formally,

    wherex,ydenote ground truth and generated image respectively, (i, j) denotes the position in the image, andmsignifies the number of total elements.The lowerL1loss means generated images are closer to ground truths for pixel-wise difference.

    (6) RMSE is used to measure the deviation between generated image and ground truth.Formally,

    wherex,ydenote ground truth and generated image respectively,(i,j)denotes the position in the image,andmsignifies the number of total elements.Similarly,the lower RMSE means generated images are closer to ground truths.

    4.4 Qualitative Results

    Figs.2 and 3 illustrate the qualitative results of different methods with 64 × 64 valid pixels input on the different datasets.It is easy to see that the original PICnet generated blurry textures and distorted structures in the outpainting areas (see Fig.2c).To solve the existing problems, we first introduce perceptual loss and style loss.For style loss, PICnet-S (PICnet with style loss) could improve the existing distorted structures,and these coarse results become much smoother(see Fig.2d).Furthermore,we used both style loss and perceptual loss in PICnet(denoted as PICnet-SP)(PICnet with style loss and perceptual loss) to improve the outpainting results.We can see the details from Fig.2e.Compared with the results of PICnet-S,PICnet-SP exhibits better on the Places2.For instance,with style loss and perceptual loss,the results are more realistic and more natural in general.To further improve the quality of outpainting images, we fuse the texture information in the low-level feature map of encoder, and simultaneously incorporate these aggregated features reusability with semantic(structural)information of deep feature map by element-wise adding in decoder.Simultaneously,we designed multi-patch discriminator into the network.This could utilize more sophisticated texture information to generate more authentic outpainting images(see Fig.2f).We can see that our PICnet-SP-LM achieved a more authentic outpainting result.Moreover,we also find a similar effect on the Paris dataset.In the Fig.3c, the vanilla PICnet method produces poor results which are filled with fuzzy contents and shadows.However, the outpainting parts are improved a lot when we add style loss alone or both style loss and perceptual loss (see Figs.3d and 3e).Specifically, these shadows disappear in some degree and the blurry textures become clearer.Fig.3f with the low-level feature fusion and the multi-patch discriminator exhibits better than the former methods.This proves that low-level feature fusion and multi-patch discriminator could promote the network to generate higherquality outpainting images.

    Figure 2:Qualitative results of different methods with 64×64 valid pixels’input on the Places2 dataset

    To further evaluate the effectiveness of our method, we set 128 × 64 valid pixels as the input of network (see Figs.4b and 5b).Figs.4 and 5 show the qualitative results of different methods on Places2 and Paris,respectively.From Figs.4c and 5c,the original PICnet produces poor outpainting results with apparent boundaries and warped structures.Nonetheless, these situations are greatly improved when we use perceptual loss and style loss.In the Figs.4 and 5,the structures become more natural and clearer(see Figs.4d,4e, 5d and 5e).Moreover,Figs.4f and 5f,generated by our PICnet-SP-LM,reach the higher effect than the others.Thus,these results once again demonstrate that both low-level feature fusion and the multi-patch discriminator are instrumental for network to improve the quality of outpainting images.

    Figure 3: Qualitative results of different methods with 64 × 64 valid pixels’ input on the Paris StreetView dataset

    Figure 4: Qualitative results of different methods with 128 × 64 valid pixels’ input on the Places2 dataset

    Figure 5: Qualitative results of different methods with 128 × 64 valid pixels’ input on the Paris StreetView dataset

    4.5 Quantitative Results

    The qualitative results of different methods on both Paris and Places2 datasets with different inputs are shown in Tabs.1–4.The quantitative results with 64×64 valid pixels’input on Paris and Places2 are shown in the Tabs.1 and 2.In the Tab.1,we exhibit the quantitative metrics of 20,000 test images on the Places2.In the experiments, our method with low-level feature fusion and the multipatch discriminator also achieves better metrics.Specially, our PICnet-SP-LM method achives the lower 30.81 for FID, signifying our model can realize clearer and more diverse outpainting results.The higher PSNR of 13.72 and SSIM of 0.4261, proving our results have a better image structure.Besides, we also obtain lower L1 loss of 34.47 and RMSE of 64.76, which indicates our results are closer to ground truths for pixel difference.Tab.2 shows the quantitative metrics on the Paris.As result of the limitation of the 100 test images of Paris,we only measure the metrics SSIM and RMSE.From the quantitative results, low-level feature fusion and multi-patch discriminator again improve the results generated by the vanilla PICnet.

    Furthermore, Tabs.3 and 4 show the quantitative results of different methods with 128 × 64 valid pixels’ input on Places2 and Paris.The effect of low-level feature fusion and the multi-patch discriminator once presents in the tables.Vanilla PICnet method produces the poor results which have lower-quality quantitative metrics.Contrarily,the quantitative metrics of outpainting results produced by PICnet-SP-LM can realize a better degree.Specially,with the effect of the low-level feature fusion and the multi-patch discriminator,PICnet-SP-LM achieves higher PSNR of 16.78 and SSIM of 0.6452 on the Places2 dataset.Meanwhile,PICnet-SP-LM also realizes the lower FID of 9.99 and L1 loss of 19.25.In addition,on the Paris dataset,PICnet-SP-LM also exhibits better for SSIM and RMSE.All the experiments demonstrate that both low-level feature fusion and the multi-patch discriminator are beneficial for outpainting network to improve the quality of outpainting images.

    Table 1:Quantitative results of different methods with 64×64 valid pixels’input on the Places2 dataset

    Table 2:Quantitative results of different methods with 64×64 valid pixels’input on Paris StreetView.Because the limitation of the 100 test images of Paris StreetView,we only evaluate the SSIM and RMSE

    Table 3: Quantitative results of different methods with 128 × 64 valid pixels’ input on the Places2 dataset

    Table 4:Quantitative results of different methods with 128×64 valid pixels’input on Paris StreetView.Because the limitation of the 100 test images of Paris StreetView,we only evaluate the SSIM and RMSE

    4.6 Ablation Study

    In addition,we also implement other experiments for further selecting the better PICnet-SP-LM method.Tab.5 is the quantitative results of implemental experiments on the Places2 dataset.Specifically, PICnet-SP-LM-1 and PICnet-SP-LM-2 are the different hyper parameters for reconstruction loss and KL loss,respectively.(PICnet-SP-LM-1 with hyper parameter 20 for reconstruction loss and hyper parameter 20 for KL loss, and PICnet-SP-LM-2 with hyper parameter 20 for reconstruction loss and hyper parameter 40 for KL loss.)From the experimental results,PICnet-SP-LM-1 achieves a better degree.Thus,PICnet-SP-LM-3 and PICnet-SP-LM-4 adopt the hyper parameters of PICnet-SP-LM-1.PICnet-SP-LM-3 utilizes one layer’s aggregated features,and PICnet-SP-LM-4 utilizes two layers’aggregated features.Apparently,PICnet-SP-LM-4 utilizing more aggregated features achieves a better effect.Therefore, PICnet-SP-LM-4 is an optimal experimental setup, which could generate more natural and more realistic outpainting results.Moreover, for the qualitative aspect, the results generated by PICnet-SP-LM-4 are also clearer and more authentic than other methods.In the Fig.6,we also select some outpainting results with borders in baseline model.Then we relieve or eliminate these borders through gradually adding our core blocks, which could present the obvious effect of these core blocks.

    Table 5: Quantitative results of ablation study with 64×64 valid pixels’input on the Places2 dataset

    Figure 6:Qualitative results of ablation study on the Places2 dataset.(a)Input,(b)PICnet-SP-LM-1,(c)PICnet-SP-LM-2,(d)PICnet-SP-LM-3,(e)PICnet-SP-LM-4

    5 Conclusion

    In fact,image outpainting plays an important role in image processing field,and it can be also used to promote the image inpainting.In this paper, we present a perceptual image outpainting method,which is assisted by low-level feature fusion and multi-patch discriminator.In details, we first fuse the low-level texture information in the encoder,and simultaneously incorporate these fused features with semantic(or structural)information of deep feature map,which could promote the network to generate finer outpainting results.At the same time, we also present a multi-patch discriminator to enhance the generated image texture,which effectively judges the generated image from the different level features and impels our network to produce more natural and clearer outpainting results.To fully evaluate our model,we implement experiments on Places2 and Paris dataset.Finally,the experimental results show that our method is better than PICnet for qualitative effects and quantitative metrics,which proves the effectiveness and efficiency of our method for image outpainting task.In the future,we will further study more challenging image outpainting field,such as the input images with bigger missing regions.We also try to realize higher-quality outpainting results.

    Acknowledgement:I would like to thank those who helped me generously in this research.

    Funding Statement:This work was supported by the Sichuan Science and Technology program(2019JDJQ0002, 2019YFG0496, 2021016, 2020JDTD0020), and partially supported by National Science Foundation of China 42075142.

    Conflicts of Interest:The authors declare that they have no conflicts of interest to report regarding the present study.

    亚洲av电影在线观看一区二区三区| 国产精品无大码| 久久久久久人人人人人| 在线观看免费高清a一片| 大片免费播放器 马上看| 2022亚洲国产成人精品| 精品国产乱码久久久久久男人| 中文字幕另类日韩欧美亚洲嫩草| 欧美日本中文国产一区发布| 天天操日日干夜夜撸| 日韩制服骚丝袜av| 免费观看无遮挡的男女| 国产成人欧美| 免费看不卡的av| 欧美激情高清一区二区三区 | 久久鲁丝午夜福利片| 久久精品国产亚洲av高清一级| 久久久久久久久久人人人人人人| 成人漫画全彩无遮挡| 人妻人人澡人人爽人人| 国产亚洲精品第一综合不卡| av有码第一页| tube8黄色片| 在线观看人妻少妇| 一区二区日韩欧美中文字幕| 青春草视频在线免费观看| 高清视频免费观看一区二区| 在线天堂最新版资源| 一级片免费观看大全| 久久99精品国语久久久| 亚洲天堂av无毛| 十分钟在线观看高清视频www| 有码 亚洲区| 搡老乐熟女国产| 久久人人爽av亚洲精品天堂| 日日摸夜夜添夜夜爱| 午夜老司机福利剧场| 一本大道久久a久久精品| 精品卡一卡二卡四卡免费| 欧美少妇被猛烈插入视频| 久久午夜综合久久蜜桃| 久久久久人妻精品一区果冻| 日韩不卡一区二区三区视频在线| 成人手机av| 国产精品 国内视频| 国产精品一国产av| 26uuu在线亚洲综合色| 久久人人爽av亚洲精品天堂| 麻豆乱淫一区二区| 久久毛片免费看一区二区三区| 亚洲欧美精品自产自拍| 国产成人91sexporn| 日日摸夜夜添夜夜爱| 欧美亚洲日本最大视频资源| 国产午夜精品一二区理论片| 边亲边吃奶的免费视频| 亚洲综合色网址| 亚洲国产最新在线播放| 日本欧美视频一区| 精品人妻在线不人妻| 边亲边吃奶的免费视频| 久久久久人妻精品一区果冻| 色婷婷久久久亚洲欧美| 久久狼人影院| 国产黄色免费在线视频| 多毛熟女@视频| 五月天丁香电影| 久久精品国产鲁丝片午夜精品| 欧美激情极品国产一区二区三区| 观看av在线不卡| 中文字幕最新亚洲高清| 欧美日韩亚洲高清精品| 90打野战视频偷拍视频| 成人国语在线视频| 一级毛片我不卡| 伊人久久国产一区二区| 久久久国产精品麻豆| 日韩伦理黄色片| 日日摸夜夜添夜夜爱| 午夜精品国产一区二区电影| 国产一区二区三区综合在线观看| 人人妻人人澡人人看| 啦啦啦在线免费观看视频4| 另类精品久久| av有码第一页| 久久久久视频综合| 免费日韩欧美在线观看| 天天影视国产精品| 亚洲精品自拍成人| 一级毛片电影观看| 韩国av在线不卡| 午夜免费男女啪啪视频观看| 欧美日韩精品成人综合77777| 观看美女的网站| 免费看av在线观看网站| 久久国产精品大桥未久av| 极品人妻少妇av视频| 色播在线永久视频| 亚洲国产精品国产精品| 中文字幕另类日韩欧美亚洲嫩草| 日本猛色少妇xxxxx猛交久久| www日本在线高清视频| 国产一区有黄有色的免费视频| 天堂中文最新版在线下载| 国产一区亚洲一区在线观看| 久久久久国产精品人妻一区二区| 久久久精品免费免费高清| 精品国产一区二区三区久久久樱花| 久久久久久伊人网av| 我要看黄色一级片免费的| 一级爰片在线观看| 亚洲精品日韩在线中文字幕| 黄色怎么调成土黄色| 久热久热在线精品观看| 丝袜美腿诱惑在线| 日韩一卡2卡3卡4卡2021年| 99九九在线精品视频| 十八禁高潮呻吟视频| 成人亚洲欧美一区二区av| 18禁裸乳无遮挡动漫免费视频| 久久国内精品自在自线图片| 欧美变态另类bdsm刘玥| 欧美日韩一级在线毛片| 亚洲四区av| 成年女人在线观看亚洲视频| 亚洲人成电影观看| av国产久精品久网站免费入址| 久久精品国产鲁丝片午夜精品| 亚洲人成77777在线视频| 亚洲一级一片aⅴ在线观看| 老司机亚洲免费影院| 久久狼人影院| 婷婷色综合大香蕉| 日日摸夜夜添夜夜爱| 女人高潮潮喷娇喘18禁视频| 丝袜美腿诱惑在线| 国产精品一区二区在线不卡| 久久久久国产精品人妻一区二区| 十八禁高潮呻吟视频| 亚洲国产日韩一区二区| 男女啪啪激烈高潮av片| 十分钟在线观看高清视频www| 亚洲欧美一区二区三区久久| 亚洲国产成人一精品久久久| 宅男免费午夜| 肉色欧美久久久久久久蜜桃| 美女主播在线视频| av免费在线看不卡| 国产综合精华液| 老司机影院成人| 精品国产超薄肉色丝袜足j| 婷婷成人精品国产| 男人添女人高潮全过程视频| 精品酒店卫生间| 久久久亚洲精品成人影院| 亚洲欧洲国产日韩| 欧美另类一区| 高清欧美精品videossex| 欧美亚洲日本最大视频资源| 午夜免费男女啪啪视频观看| 男女下面插进去视频免费观看| 菩萨蛮人人尽说江南好唐韦庄| 人人妻人人添人人爽欧美一区卜| www日本在线高清视频| 极品少妇高潮喷水抽搐| 亚洲男人天堂网一区| 国产欧美日韩综合在线一区二区| 中文乱码字字幕精品一区二区三区| 蜜桃在线观看..| 欧美日韩视频精品一区| 国产免费一区二区三区四区乱码| 欧美变态另类bdsm刘玥| 亚洲综合色惰| 日韩一区二区三区影片| 亚洲av电影在线进入| av免费在线看不卡| 久久久久久免费高清国产稀缺| 捣出白浆h1v1| 国产成人精品婷婷| 99久久精品国产国产毛片| 欧美 日韩 精品 国产| 亚洲精品久久成人aⅴ小说| av视频免费观看在线观看| 亚洲av中文av极速乱| 免费观看无遮挡的男女| 80岁老熟妇乱子伦牲交| 如日韩欧美国产精品一区二区三区| 中文字幕av电影在线播放| av在线老鸭窝| freevideosex欧美| av卡一久久| 欧美精品高潮呻吟av久久| 天天操日日干夜夜撸| 好男人视频免费观看在线| 国产精品久久久久久精品古装| 成年人免费黄色播放视频| 精品一区二区三区四区五区乱码 | 午夜福利,免费看| 如日韩欧美国产精品一区二区三区| 精品第一国产精品| 丝袜脚勾引网站| 男人舔女人的私密视频| 亚洲国产精品999| 美女大奶头黄色视频| 可以免费在线观看a视频的电影网站 | 亚洲伊人色综图| 国产成人一区二区在线| 亚洲av日韩在线播放| 日韩制服骚丝袜av| 国产免费视频播放在线视频| 另类精品久久| 国产午夜精品一二区理论片| 亚洲成色77777| 久久精品国产自在天天线| 久久精品国产综合久久久| 国产日韩欧美在线精品| 制服诱惑二区| 国产精品熟女久久久久浪| 深夜精品福利| 爱豆传媒免费全集在线观看| 老汉色∧v一级毛片| 最近中文字幕2019免费版| 2018国产大陆天天弄谢| 天堂俺去俺来也www色官网| 91精品三级在线观看| 母亲3免费完整高清在线观看 | 侵犯人妻中文字幕一二三四区| 午夜免费观看性视频| 观看av在线不卡| 欧美 亚洲 国产 日韩一| 女人高潮潮喷娇喘18禁视频| 日产精品乱码卡一卡2卡三| 国产av国产精品国产| 极品少妇高潮喷水抽搐| 哪个播放器可以免费观看大片| 热re99久久国产66热| 日韩视频在线欧美| 日本wwww免费看| 女性被躁到高潮视频| 免费在线观看完整版高清| 国产一区二区在线观看av| 亚洲精品第二区| 赤兔流量卡办理| 国产成人精品久久久久久| 欧美日韩视频高清一区二区三区二| 国产精品久久久久久久久免| 麻豆av在线久日| 国产免费视频播放在线视频| 黄片小视频在线播放| 久久久精品国产亚洲av高清涩受| 久久久精品区二区三区| 寂寞人妻少妇视频99o| 日本爱情动作片www.在线观看| 91成人精品电影| 亚洲精品,欧美精品| 亚洲国产欧美网| 少妇精品久久久久久久| 五月伊人婷婷丁香| 欧美成人午夜精品| 国产福利在线免费观看视频| 毛片一级片免费看久久久久| 精品人妻熟女毛片av久久网站| 美女国产视频在线观看| 国产欧美日韩一区二区三区在线| 日产精品乱码卡一卡2卡三| 亚洲精品国产av成人精品| 亚洲欧美成人精品一区二区| 色哟哟·www| 午夜精品国产一区二区电影| 日本av免费视频播放| 在现免费观看毛片| 国产有黄有色有爽视频| 久久精品久久精品一区二区三区| 欧美激情 高清一区二区三区| 中国三级夫妇交换| 自线自在国产av| 日韩av不卡免费在线播放| 91精品伊人久久大香线蕉| 亚洲欧美成人精品一区二区| 国产成人精品婷婷| 国产亚洲av片在线观看秒播厂| 国产精品无大码| 亚洲精品日本国产第一区| 久久久久久久久久久免费av| 久久热在线av| 免费看不卡的av| 亚洲一级一片aⅴ在线观看| 久久久久久久大尺度免费视频| 男女下面插进去视频免费观看| 午夜福利视频精品| 中文字幕人妻丝袜一区二区 | 国产在视频线精品| 国产成人精品福利久久| 久久热在线av| 午夜免费男女啪啪视频观看| 多毛熟女@视频| 97在线人人人人妻| 亚洲欧洲国产日韩| 婷婷成人精品国产| 免费高清在线观看视频在线观看| 久久精品夜色国产| 大话2 男鬼变身卡| 我要看黄色一级片免费的| 国产欧美亚洲国产| 久久精品久久精品一区二区三区| 伦精品一区二区三区| 免费久久久久久久精品成人欧美视频| 新久久久久国产一级毛片| 青草久久国产| 人妻少妇偷人精品九色| 黄片小视频在线播放| 国产精品秋霞免费鲁丝片| 成年人午夜在线观看视频| 捣出白浆h1v1| 精品第一国产精品| 丝瓜视频免费看黄片| 国产免费福利视频在线观看| 黄色毛片三级朝国网站| 少妇精品久久久久久久| 国产精品成人在线| 国产精品久久久久久精品电影小说| 人体艺术视频欧美日本| 99久久综合免费| 亚洲男人天堂网一区| 久久精品国产自在天天线| 日本av手机在线免费观看| 久久久久久久亚洲中文字幕| 国产一区二区激情短视频 | 精品一区在线观看国产| 婷婷色av中文字幕| 亚洲图色成人| 久久久久久久久久人人人人人人| 少妇人妻久久综合中文| 亚洲欧美中文字幕日韩二区| 一本一本久久a久久精品综合妖精 国产伦在线观看视频一区 | 色哟哟·www| 欧美 日韩 精品 国产| 国产成人精品久久久久久| 青青草视频在线视频观看| 999久久久国产精品视频| 欧美人与善性xxx| 国产麻豆69| 亚洲精品日本国产第一区| 人人妻人人爽人人添夜夜欢视频| 久久精品久久久久久噜噜老黄| 亚洲精品日本国产第一区| 1024香蕉在线观看| 26uuu在线亚洲综合色| av电影中文网址| 亚洲第一青青草原| 成人国产麻豆网| 欧美人与性动交α欧美软件| 热99久久久久精品小说推荐| 午夜av观看不卡| 咕卡用的链子| 青春草视频在线免费观看| 如日韩欧美国产精品一区二区三区| 中文乱码字字幕精品一区二区三区| 免费高清在线观看日韩| 91精品伊人久久大香线蕉| 不卡视频在线观看欧美| 看免费av毛片| 岛国毛片在线播放| 看免费av毛片| 国产精品.久久久| 色视频在线一区二区三区| 精品一区二区三卡| 日韩三级伦理在线观看| 一级爰片在线观看| 高清av免费在线| 精品亚洲乱码少妇综合久久| av片东京热男人的天堂| 99久久人妻综合| 欧美日韩视频精品一区| 丝袜在线中文字幕| 欧美日韩精品成人综合77777| 热re99久久精品国产66热6| 这个男人来自地球电影免费观看 | 男女午夜视频在线观看| 女人精品久久久久毛片| 极品少妇高潮喷水抽搐| 亚洲欧美成人综合另类久久久| 人妻少妇偷人精品九色| 久久国产精品大桥未久av| 国产精品女同一区二区软件| 久久 成人 亚洲| 人妻少妇偷人精品九色| 看十八女毛片水多多多| 777米奇影视久久| 亚洲精品中文字幕在线视频| 久久女婷五月综合色啪小说| 男女边吃奶边做爰视频| 熟妇人妻不卡中文字幕| av网站免费在线观看视频| 国产1区2区3区精品| 日韩一区二区视频免费看| 亚洲av成人精品一二三区| 国产精品久久久久久精品电影小说| 美女高潮到喷水免费观看| 熟女少妇亚洲综合色aaa.| 18禁观看日本| 午夜福利网站1000一区二区三区| 美女福利国产在线| 久久久欧美国产精品| 日韩av免费高清视频| 在线观看免费视频网站a站| 一边亲一边摸免费视频| 午夜福利一区二区在线看| 国产激情久久老熟女| 成年动漫av网址| 久久久国产一区二区| 精品一品国产午夜福利视频| 有码 亚洲区| 国产午夜精品一二区理论片| 日韩一卡2卡3卡4卡2021年| 999久久久国产精品视频| 99精国产麻豆久久婷婷| 一本大道久久a久久精品| 欧美最新免费一区二区三区| 日韩免费高清中文字幕av| av.在线天堂| av电影中文网址| 中文字幕人妻丝袜制服| 欧美日韩精品网址| 91国产中文字幕| 另类精品久久| 国产精品秋霞免费鲁丝片| 亚洲av在线观看美女高潮| 美女国产视频在线观看| 天天影视国产精品| 超碰成人久久| 亚洲精品久久成人aⅴ小说| 国产精品成人在线| av在线播放精品| 男女无遮挡免费网站观看| 久久这里只有精品19| 久久精品夜色国产| 90打野战视频偷拍视频| 99热国产这里只有精品6| 一级毛片我不卡| 高清视频免费观看一区二区| 亚洲精品国产一区二区精华液| 免费黄网站久久成人精品| 18在线观看网站| 日本欧美国产在线视频| 在线天堂中文资源库| 亚洲欧洲精品一区二区精品久久久 | 极品人妻少妇av视频| 国产成人欧美| 日韩av在线免费看完整版不卡| 国产色婷婷99| 久久久久久久大尺度免费视频| 欧美xxⅹ黑人| 丰满少妇做爰视频| 亚洲av电影在线进入| 午夜福利视频精品| 欧美激情极品国产一区二区三区| 久久人人爽av亚洲精品天堂| 日韩电影二区| 亚洲国产av影院在线观看| 飞空精品影院首页| 亚洲激情五月婷婷啪啪| 一本久久精品| 国产一区二区激情短视频 | 丝袜美足系列| 欧美+日韩+精品| 久久久精品94久久精品| 亚洲国产毛片av蜜桃av| 十分钟在线观看高清视频www| 亚洲美女搞黄在线观看| 国产精品秋霞免费鲁丝片| 亚洲精品国产av蜜桃| 18在线观看网站| 狠狠精品人妻久久久久久综合| 男人操女人黄网站| 久久久久国产精品人妻一区二区| 精品一区二区三卡| 国产又色又爽无遮挡免| 人人妻人人澡人人看| 黄色 视频免费看| 丝袜人妻中文字幕| www日本在线高清视频| 美女高潮到喷水免费观看| 国产一区二区 视频在线| 日韩精品免费视频一区二区三区| 最新中文字幕久久久久| 国产亚洲一区二区精品| 免费高清在线观看视频在线观看| 亚洲国产色片| 免费观看av网站的网址| 久久精品久久精品一区二区三区| kizo精华| 国产人伦9x9x在线观看 | av免费观看日本| 久久久精品区二区三区| 久久久亚洲精品成人影院| www日本在线高清视频| 成人亚洲欧美一区二区av| 亚洲,一卡二卡三卡| 熟女av电影| 欧美另类一区| 久久精品aⅴ一区二区三区四区 | 777久久人妻少妇嫩草av网站| 大话2 男鬼变身卡| 一本久久精品| 1024视频免费在线观看| 国产精品一二三区在线看| 美女主播在线视频| 亚洲第一av免费看| 亚洲情色 制服丝袜| 色网站视频免费| 久久久久精品性色| 在线观看免费日韩欧美大片| 极品少妇高潮喷水抽搐| 一级毛片电影观看| 女性生殖器流出的白浆| 欧美日韩国产mv在线观看视频| 日韩人妻精品一区2区三区| 中文字幕制服av| 性少妇av在线| 成人午夜精彩视频在线观看| 色视频在线一区二区三区| 欧美激情高清一区二区三区 | 熟女av电影| 大片免费播放器 马上看| 电影成人av| 18禁动态无遮挡网站| 亚洲av福利一区| 久久av网站| 午夜91福利影院| 亚洲精品日韩在线中文字幕| 欧美成人午夜免费资源| 一级毛片电影观看| 在线天堂中文资源库| 两个人看的免费小视频| 亚洲欧美成人精品一区二区| 91久久精品国产一区二区三区| 国产探花极品一区二区| 国产精品嫩草影院av在线观看| 久久99蜜桃精品久久| 在线观看人妻少妇| 人人澡人人妻人| 国产av码专区亚洲av| 青春草国产在线视频| 菩萨蛮人人尽说江南好唐韦庄| 另类亚洲欧美激情| 亚洲成人手机| freevideosex欧美| 国产亚洲精品第一综合不卡| 久久久久人妻精品一区果冻| 国产av码专区亚洲av| 99热全是精品| 最近最新中文字幕免费大全7| 久久亚洲国产成人精品v| av国产久精品久网站免费入址| 成人毛片60女人毛片免费| 香蕉精品网在线| 一级片'在线观看视频| 黄色一级大片看看| 欧美国产精品va在线观看不卡| 哪个播放器可以免费观看大片| 国产日韩欧美在线精品| 国产激情久久老熟女| 国产午夜精品一二区理论片| 日韩欧美精品免费久久| 中文欧美无线码| 涩涩av久久男人的天堂| 人人澡人人妻人| 免费女性裸体啪啪无遮挡网站| 老汉色∧v一级毛片| 日韩三级伦理在线观看| 在线天堂中文资源库| 777久久人妻少妇嫩草av网站| 久久久久精品久久久久真实原创| 欧美变态另类bdsm刘玥| 久久午夜福利片| 丝袜人妻中文字幕| 国产日韩欧美亚洲二区| 欧美日韩视频高清一区二区三区二| 国产成人精品久久久久久| 国产精品久久久久久久久免| 午夜久久久在线观看| 欧美日本中文国产一区发布| 水蜜桃什么品种好| 99热网站在线观看| 18+在线观看网站| 高清欧美精品videossex| 如日韩欧美国产精品一区二区三区| 美女午夜性视频免费| 啦啦啦在线免费观看视频4| 伊人亚洲综合成人网| 乱人伦中国视频| 在线观看国产h片| 亚洲欧美一区二区三区黑人 | 美女国产视频在线观看| 深夜精品福利| 五月天丁香电影| 日韩精品有码人妻一区| 人妻 亚洲 视频| av国产久精品久网站免费入址| 亚洲美女黄色视频免费看| 一级爰片在线观看| 天美传媒精品一区二区| 国产精品.久久久| 老司机亚洲免费影院| 美女主播在线视频| 一区二区日韩欧美中文字幕| 午夜91福利影院| 天天操日日干夜夜撸| 蜜桃国产av成人99| 国产麻豆69| 欧美人与善性xxx| 国产精品99久久99久久久不卡 | av电影中文网址| 波多野结衣一区麻豆|